Topography processor system

ABSTRACT

A topography processor system comprising a phased image sensor array, at least one processor arranged to perform range decompression of imaged detail, and a means for allowing non interruptive graphic macro diagnosis based on a graphic display of the frame rate macroscopic behavior of internal transfer functions of the system figures  15, 17  permit visual, augmented visual or automatic visual determination of the system&#39;s integrity.

TECHNICAL FIELD OF THE INVENTION

This invention relates to a topography processor system.

ANALYSIS OF BACKGROUND ART AND SUMMARY OF INVENTION

The advent of three dimensional optical mapping systems, based oncorrelated information from a phased array of image sensors (CCD orequivalent) sets new levels of complexity in system architecture design.This complexity poses considerable problems not only during experimentaldevelopment and integration but also sets minimum diagnosticrequirements for “first line” monitoring of a production equipment'sstatus. The purpose of this invention is the combination of featuresnamely, the architecture of a frame rate topography processor system,use only of such a system's sufficient control, data and addresshighways, and the introduction of end to end non interruptive graphicmacro diagnostic techniques to allow “first line” system GO/NOGOdecisions to be reached without the need for an extensive second layermicroscopic BITE, or the use of additional external test equipments.These capabilities provide non interruptive visual, augmented visual orautomatic visual diagnostic determination of correct end to end systeminternal parameter, or combined parameter performance characteristics.

The division between the embodiment of functionality in the hardware orsoftware of an optical typography processing systems is dictated by thetarget applications required maximum real time response. Using framestores and time dilation many systems take advantage of processors,operating within Von Neumann architectures, to identify vectors ofparticular attributes, and correlate such vectors between theirdifferent perspective images in the calculation of the system relativerange to elements of topographical detail otherwise referred to as imagedetail range decompression.

Such systems benefit from the reliability of today's digital technologyand the flexibility afforded by programming languages. The comfortafforded by high level software exacts a price for their inherentexpansion factors in the serial execution speed of such processes, andthe reduced visibility of the target machine process mapping, andtherefore of its executed performance. Faster execution and visibilitysupported by low level software design alas remains a specialist domainin whose absence reliability and maintainability issues often arise.

For a modest system comprising three virtual imager systems withoperating bandwidths of around 5 mHZ and utilising vectors of twodifferent attributes, then it is arguable that for complete image detailrange decompression at the image sensor frame rate, would necessitatevector intercepts to be calculated at an effective rate of around 300mHZ. The latency of such processes embedded in software will currentlygenerally considerably exceed that of a frame period. Diagnosticmonitoring of software performance usually necessitates a combination ofoffline postmortem scrutiny, and or the use of monitoring by utility andtrace systems. The information from such processes is generallymicroscopic and representative of implementation computationalanomalies, or possible evidence of a failure in the passing of controlor data. In any event the provision of diagnostic information, and orits recovery from a system introduces overheads to the targetapplication, in respect of application non essential, or additionalexternal equipments, and or in respect of the latency of theirassociated serial processing techniques which further takes the systemout of its normal operational envelope, and almost certainly furtheraway from frame rate execution.

The identification and rectification of hardware failures tends to fallinto a hierarchy of approaches. Systems with integrated BITE start ofday, or continuous BITE often allow at the “first line” timelyidentification of system malfunctions. The use of such techniques toidentify GO/NOGO situations support high levels of overall systemoperability. At the “second line” intermediate level, equipmentsisolated from a system lend themselves to analysis by dedicated testequipments, and or board swapping to rectify failures. For localizedfunctional anomalies, “third line” specialist monitoring at board levelsupports rectification of failed components.

Characteristic of all these techniques for both software and hardware isthat they address microscopic detail in a system's performance, whoseinterpretation may provide evidence, in specific contexts, of the causalmechanisms for failure of macroscopic functionality. However many of thetechniques tend to clinically isolate the functionality under scrutiny,and therefore isolate some faults from the operational envelope in whichthey occur. The purpose of non interruptive graphic macro diagnostics isto achieve through envelope testing, the rapid identification of systemfunctional GO/NOGO indications enabling if necessary part, or wholesystem removal for “second” or “third line” testing.

For what may recently be considered as a complex system in which aninterprocessor link supports 1000 parameters, optimistically, forprocess completion within a frame period, image detail rangedecompression processing systems need to process two to three orders ofmagnitude more data, realistically 300,000 parameters within the frameperiod. A topographical mapping system architecture described latercapable of correlating all of the imaged generic pattern data within theframe period of a phased image sensor array, comprising a minimum ofthree virtual image sensors operating around 5 mHZ, has overall systemdata requirements in the order of 600 Million units (such systems tendto operate with mixed word lengths) of partitioned online RAM. Anexperimental implementation of an element of such a processing systemcomprising a single line scan processor necessitated some 60 Euro cardscontaining in all some 700 integrated circuits of which about 10% wereVLSI and a guesstimated 20,000 pin to pin threads. To support continuousprocessing of data at the frame rate some 30 simultaneous paralleladdress processes, in support of the machines inherent multiple addresscapabilities, operate within each line scan processing element at themachine's clock rate.

Whilst physically such an experimental system lends itselftechnologically to miniaturization the complexity remains, and thereforealso a clear requirement not only for the use of diagnostics in supportof system development and integration, but also in support of the use ofsuch production equipments. Whilst many systems benefit from afunctionally integrated level of BITE allowing the stimulation andmonitoring of system control and data threads, the introduction of asecond layer of circuitry here poses immediate problems of scale.Further in practical terms the cost of such an integration for a“microscopic” diagnostic capability suggests a different approach toGO/NOGO system integrity is better sought, allowing the development ofcheaper special to type “second” and “third line” test equipments.

Optical topography processor systems are by nature concerned with imageddata, which we (humans) can quickly asses. In the context ofdiagnostics, graphic macro diagnostics allow such human visual, oraugmented visual macroscopic assessment of end to end internal systemperformance, or the automatic assessment of system performance based onsuch visual patterns.

For a topography processor system operating synchronously at the framerate of an image sensor, advantage may be taken of a graphic macrodiagnostic capability in the analysis of internal as well as overallsystem performance characteristics. Equally for such a system withinternal asynchronous functionality, executing and completing within aframe period, then for a defined apriori system drive such asynchronousfunctionality may also be considered to be operating in a synchronousfashion, and therefore also lends itself to the same graphic macrodiagnostics techniques. An example of an analogous capability is thetelevision test card where macroscopic assessment of the overall end toend performance of an equipment can be easily made using only thefunctionality of the equipment under test, and of course the assumedintegrity of the broadcast test picture. Topographical mapping systems,particularly those employing general purpose processors to effect rangedecompression of image detail, generally allow display monitoring of theimaged scenario, but thereafter time dilated processing of imaged datatends towards asynchronous manipulation of binary representations of theimaged scenario. The reconstitution and display of meaningful images insuch a system, particularly of intermediate process results,necessitates interruptive system processing if only to allow sufficientserial processing time for the generation of what may often only be asnapshot.

For a frame rate topography processor system as outlined above, avariety of possibilities exist to monitor non interruptively systeminternal and overall performance characteristics utilising the hardwarededicated to the system's prime functionality. This may be achieved bymixing process parameters with appropriate frame and line sync in thegeneration of graphic macro diagnostic information for display on thesystem's prime display. This can include all stages of the system'sprocesses, end to end, from imaged scenario and field of regard scanningassessment and calibration, through to performance of image threadprocessing, input of compressed data to partitioned input stacks wherethe asynchronous write enable commands and digit output of stackpointers generate graphic macro diagnostic indications of processorinput performance and address integrity for each channel of the system.Further the correlation of imaged information in the calculation ofvector intercepts for pairs of channels and specific vector attributesare equally inherently available at the frame rate, for visualmonitoring, for each of the various continuous parallel processors. Thefurther characteristics of line processor iteration rates, and the ratesof line process completion also lend themselves to graphic macrodiagnostic analysis. Intermediate computation results of multipleaddress transform processes defining vector pair intercepts aresimilarly monitorable. The combination of vector pair intercepts in thegeneration of each line processor's parallel frame rate output of thesystem scenario data base is similarly possible.

Various possibilities exist to contrive particular visual patterns, sets(discontinuous in time) of sets of data continuous in time, in thestimulation of such a system, whereby aspects of the systems performancerepresented by their associated graphic macro diagnostic response, sets(discontinuous in time) of sets of data continuous in time, may bemonitored. It is clearly a function of the various system transferfunctions and the definition of the stimulation data sets, as to theform of the graphic diagnostic response. Consideration of particularsystem transfer functions and a suitably appropriate definition ofstimulation data sets, support the ergonomic definition of graphicdiagnostic visual response patterns. Where the response pattern data ofparticular system parameters makes ergonomic design difficult, augmentedvisual diagnostic responses may be employed. Here for an aprioristimulation, display of the difference or error graphical macrodiagnostic response indicating departure from the system anticipatedresponse may be generated. Referring back to the analogy of thetelevision test card, such a display could in the case of a televisionindicate those parts of the television's transfer characteristics whichfall outside the anticipated performance specification. The techniquesfurther do not restrict themselves to the monitoring of singleparameters but allow, for an apriori system drive, the simultaneousmonitoring of for example complete address generation through the use oftransform processors permitting the display of again an error ordifference signal.

A number of different approaches support the generation of timedistributed image pattern data, sets (discontinuous in time) of sets ofdata continuous in time, in support of the graphic macro diagnosticmonitoring of system internal process performance characteristics. Onetechnique would be to physically introduce into the system's imagesensor optical systems graphical images (perhaps photographs of the realworld) of the different perspectives of the individual imagerscomprising the phased array, or to introduce separate video recordingsof such images into the output channels of the image sensors. In theformer case some host environments would not necessarily lend themselvesto the mechanical interference associated with the introduction ofphysical images into the imagers optical systems. In the latter case theaccurate external dynamic control of a video recorder's frame and linesync generation non interruptively in sympathy with the system's videosyncing demands, unlike that of an image sensor, pose considerableproblems.

It is suggested that for a frame rate topographical mapping systemincluding the type described later that the introduction of minimal noninterruptive hardware to effect system stimulation and system diagnosticresponse monitoring fed to the system video mixer can support acomprehensive range of system GO/NOGO integrity checks. In particular adifferential binary event generator permits stimulation of thetopography processor system under manual or automatic control where thetime relative generation of such binary events (vectors) between theprocessor's binary event input channels allows the simulation of threedimensional spacial reference points enabling manual system end to endconfidence checks to be made. Further visual, augmented visual orautomatic visual system integrity checks may be made for apriori threedimensional reference points, implied by specific differential vectors,injected into the system where each stage of their processing generatesdeterminable results for comparison allowing in addition to staticsystem integrity checks, the further assessment of the system's dynamicresponse characteristics. Extended analysis of other design aspects forexample the integrity of internal address generation may be made byswitching on line a variety of identity transform processors whosediffering transform characteristics allow the visual, augmented visualor automatic isolation of system functional or component anomalies,whilst non interruptively preserving the overall integrity of thesystem's end to end architecture.

The example of a frame rate generic pattern derived topography processorused in the main line description is based on the premise that for agroup of three or more image sensors whose fields of view share a commonscenario, then within this common scenario the spacial position of anelement of topographical detail is represented by a unique multiplevector intersection comprising one vector from each image sensor. Inparticular and simplifying the problem then for a group of three or morelogically and physically aligned image sensors gaining separateperspective views of the same scenario, such that common scenario imagedetail is contrived to register in corresponding line scans of each ofthe image sensors in the group, then if sets of data elements havingsimilar attributes, excepting their position within a particular linescan and which data may further be considered as representative ofvectors from their host image sensor to elemental scenario detail, canbe identified from each image sensor's composite video signal then theassociation between such vectors, contained within sets of vectors froma particular image sensor with vectors contained in similar sets fromthe other image sensors, can be made by considering all possiblecombinations of vectors between such sets of sets, including one fromeach image sensor, where the existence of combinations of such vectorshaving a common multiple and real intersection resolves the genericpattern recognition problems of vector association and the spacialpositioning of the topographical detail in the observed scenario.

A system architecture and processes capable of supporting such framerate generic pattern recognition, that is vector identification andassociation whereby the spacial resolution between members of sets ofsets of such vectors, fundamental to automatic topographical mapping ofthis kind, may be described by considering three distinct areas offunctionality which may in practice constitute sub systems.

Firstly a sub system comprising a minimum number of three image sensorsorganised as a phased image sensor array, in which the image sensors canbe slaved to give different perspective views of a common scenario andwhere the position and orientation of each image sensor is such that theboresights of their individual fields of view are parallel, and commonimage detail is registered in corresponding line scans of each of theimage sensors, that is at common angles of elevation within each imagesensors respective field of view, and where frame and line sync signalsof all the image sensors respective composite video signals havecorrespondence in time.

Depending on the nature of the application such a system should includethe possibility of a scanner allowing electronic as well as mechanicalscanning of the scenario by the image sensors for three main reasons,firstly to achieve angular resolutions of less than 50 micro radians andthe ranging possibilities afforded by such resolution, the angularpositioning of such a system does not lend itself entirely to mechanicalslaving. One aspect therefore of electronic slaving is to allowcontrolled slaving to these accuracies, secondly the field of view atsuch resolution for a single sensor is small therefore scanning allows apractical field of regard to be employed, thirdly the nature of thistype of ranging from aligned image sensors is such that sets of vectorpair intersections are parabolic and logarithmic in nature, andtherefore a rotation of the field of view allows better rangediscrimination particularly at extreme ranges.

Secondly a sub system comprising an image pattern thread processor orequivalent capable for each image sensor, comprising the phased imagesensor array, of simultaneously and in real time processing thecomposite video signal generated by each such image sensor to extractsets of vectors with specific attributes between these image sensors,and further time log the occurrence of all such vectors partitioned byimage sensor, attribute, and line scan (elevation angle within the imagesensor field of view) so identifying their position within the line scan(azimuth angle within the image sensor field of view). No limit is seton the number of different vector attributes to be identified, nor onthe partitioning of such sets necessary to support real time computationin the processing sub system.

Thirdly a processing sub system is necessary capable of calculatingwithin the frame period the existence of all possible real and virtualvector intersections necessary to identify multiple common and realintercepts, including one vector from each image sensor, in resolvingthe vector association and spacial positioning of the scenariostopographical detail. To achieve the effective processing ratesnecessary to resolve all possible multiple intersections of unassociatedvectors which have been automatically selected according to a particularattribute from a number of image sensors composite video signals in realtime, and thereby resolve the association of members of such sets ofsets of vectors between image sensors requires a processor architecturesupporting partitioned and parallel processing. Further a requirementexists to automatically synthesise, and again in parallel, theidentities of all possible combinations of pairs of vectors betweenimage sensors, each such pair comprising a vector taken from a set ofvectors considered as from a reference image sensor and a vector takenfrom each of the sets of sets of vectors of similar attributes for eachof the other image sensors. For the pairs of vector identities sosynthesised and in parallel the architecture also requires an effectivemultiple address capability which allows the vector pair identities tosynthesise the identity of the solutions to complex mathematicalprocesses, where the finite apriori knowledge concerning the existenceof possible vector intersections or other processes, permits thedefinition of identity transforms representing the result of suchprocesses on particular pairs of identities, that is a capability tosynthesise a third identity from a particular pair of identities. Amultiple address capability in the conventional sense allows informationbeing processed to be moved from the contents of one address to thecontents of another address, here the data of the input operandssubmitted for processing is implicit in their combined address identityand the process transforms this identity to produce a result implicit asa third identity. The identity transforms should be capable of multipleparallel operation to process other simultaneously synthesised vectoridentity pairs from other sets of sets of vectors, or to address anecessary precision or aspect of a particular transform. Such transformsshould also be capable of being cascaded to allow the interaction ofother variables or results of previous transforms.

The final identities from a single, parallel or cascaded transformprocess or processes forms the address identity for an ordered vectorpair intersection buffer into which the binary existence of a processresult may be written, one such buffer being dedicated to each pair ofsets of vectors. In this way simultaneous vector pair intersections canbe synthesised within the effective multiple addressing time. Bysynchronous parallel reading of sets of ordered vector pair intersectionbuffers the simultaneous event of the existence of a real vectorintersection being read from each of the dedicated buffers comprising aset, satisfies the multiple vector intersection premise for determiningthe spacial position of an element of topographical detail.

It is possible to identify specific aspects of such system'sinfrastructure, which generally support the previously outlined framerate range decompression of imaged detail. Outside of the main linedescription of an example of a topography processor system, a number ofsub system descriptions are included which amplify these and otheraspects of technologies employed. These examples generally includeelements of generic functionality already introduced but configured soas to isolate the aspect of technology under discussion. A furtherexample of an image detail range decompression system, is described,with largely asynchronous characteristics but which may in some modes ofoperation, also be considered as a synchronous system and thereforecapable of supporting the macro diagnostic analysis.

Further detailed descriptions are also included of certain other aspectsof sub system functionality, these include an example of a phased imagesensor array, such an array is necessary in supporting simultaneouscorrelation of differing perspective images. Similarly virtual imagesensors which support the rapid and accurate electronic positioning ofthe sensed image field of view boresight are also described in somedetail. For some operating environments three axis stabilisation ofimage sensor boresights is essential, and a further detailed descriptionin included for compatible electronic functionality to provide this.Finally an image pattern thread processor is also separately describedin some detail, such functionality is capable of identifying vectors ofparticular and different attributes at an image sensor's data rate.

An example of an iterative subset pattern derived topography processorwith largely asynchronous process characteristics is included in thetext, not least for comparative analysis, but also for its own variationof a data rate range processor. This subsystem is based on the premisethat the visual information registered in a line scan of an image sensormay be considered as a set of subset patterns. Each subset patterncomprises a set of pattern elements, where such a set of patternelements represents a sectioned element of topographical detail in theobserved scenario. The set, comprising every line scan in a frame, ofsets of subset patterns, contained in each line scan, is the set of allthe imaged topographical detail.

For two similar image sensors of known separation and orientation whosefields of view share a common scenario and where common image detail iscontrived to register in corresponding line scans of both image sensors,then from their different perspectives the spacial position of elementsof topographical detail within the common scenario is determined by theimage sensors fields of view boresight relative azimuth and elevationangles of unique pairs of pattern elements, one from each image sensorcontained within associated subset patterns.

For the image sensor pair and scenario described above the position anddefinition of the subset patterns contained within line scans of eachimage sensor will vary according to the geometry of the scenario andrelative position and orientation of each image sensor.

The automatic identification from one image sensor of pattern elementscomprising subset patterns, without recourse to models of potentialpatterns, and the further correlation of members of sets of such subsetpatterns once identified with members of sets of subset patternsdetermined by the perspective of the other image sensor poses a numberof problems.

This sub system addresses the strategies for the identification ofsubset patterns from one image sensor, and the correlation of members ofthese subset patterns with members of associated subset patterns fromthe other image sensor.

Image detail contained within the luminance signal from an image sensormay be considered to be partitioned or punctuated by the binary eventscomprising the output of an image pattern thread processor capable ofproviding pattern outline and relief contour detail of an observedscenario in real time. The binary events correspond to luminance signalfrequency excursions through preset upper or lower frequency limits.

If pairs of sets comprising luminance signal elements with the differingattributes amplitude and time differential, characterising in each setunassociated pattern elements, are generated in real time for each linescan for each of the two image sensors then such sets may further bepartitioned into subsets, characterising subset patterns, where for aparticular image sensor, and line scan a new pair of subset patterns,one for each luminance attribute, is initiated by the occurrence of aluminance frequency excursion through a preset upper or lower frequencylimit. Such pairs of subset patterns each comprising sets of elements ofa particular and different luminance attribute from one image sensor maybe compared with corresponding sets of pairs of such subset patterns(same line scan, same start criterion upper or lower frequency excursionevent, and same luminance attribute amplitude or time differential) fromthe other image sensor. The number of elements considered for comparisonbetween subset patterns is limited by the minimum number of members ofeither subset pattern considered for comparison.

For the two image sensors, as described previously and where theboresights of their respective fields of view are parallel and wheretheir frame and line sync generation is controlled such that timecoincidence exists between the characteristics of these signals betweenimage sensors then a subset pattern from the common scenario registeredby both image sensors will exist for the right hand image sensor withtime coincidence or earlier in a frame's line scan than thecorresponding associated subset pattern for the left hand image sensor.

For a dual image sensor scenario as outlined above the premise on whichthis method is based is that given a criterion by which subset patternsmay be identified then for combinations of potentially similar (samestart criterion frequency excursion event through upper set frequencylimit or (exclusive) lower set frequency limit) subset patternssynthesised for corresponding line scans from both image sensors thenfor the multiple condition of pairs of pairs of subset patterns havingequality (to a given precision) between corresponding luminanceamplitude members from one pair of subset patterns and equality (to agiven precision) between luminance time differential members from theother pair of subset patterns then a reasoned degree of probabilityexists that such members between such pairs of subset patterns betweenboth image sensors represent unique vectors having an intersection at anelement of topographical detail in the real world.

The system relative spacial position of such an element of associatedtopographical detail may be determined since the slant range isresolvable as a function of such members normally defined azimuth angleswithin each image sensors field of view. For slant ranges so calculatedthe system boresight relative height of the element of topographicaldetail is determined as a function of such members line in frame that isthe common elevation angle of the members within the image sensorsfields of view.

A variety of possibilities exist to iteratively generate combinations ofpairs of pairs of subset patterns between image sensors to allowcomparison between pairs of subset pattern members of the same attributein real time.

Having identified the subset patterns in real time for both imagesensors and passed such information of their partitioned data sets to anexternal computer system during one frame period, synthesis of subsetpattern combinations and iterative comparison of their members may bemade by software during the subsequent frame period.

For image sensors with time coincidence of their frame and line syncgeneration, hardware comparison of pairs of pairs of subset patterns maybe made in real time between combinations of pairs of pairs of subsetpatterns synthesised by the iterative (on an inter frame basis) relativemechanical slaving in azimuth of the image sensors fields of view suchthat an effective relative sweep of the observed scenario by one imagesensor's field of view in relation to the other image sensor's field ofview brings time correspondence between particular and different pairsof pairs of subset patterns contained within corresponding line scans ofthe two image sensors.

For image sensors whose fields of view boresights are fixed and wheretheir frame and line sync characteristics have default time coincidencethen the iterative, on an inter frame basis, time relative shifting offrame and line sync separation between image sensors (effective timeadvance of the lefthand image sensor's sync in relation to the righthandimage sensor sync) over a maximum of one line scans luminance periodwill also synthesise combinations of pairs of pairs of subset patternswith time coincidence between the image sensors so allowing real timehardware comparison of such pairs of pairs of subset patterns betweenimage sensors.

Co-operatively slaveable phased virtual image sensor arrays feature inboth topography processing systems described here. They supportapplications requiring that image sensors CCD or equivalent be logicallyand physically positioned and oriented, such that they generatedifferent perspective views of a common scenario, and that common imagedetail is registered in corresponding line scans of all such imagesensors in the array, and further that the frame and line syncgeneration by all such image sensors has time coincidence.

It may, for such sub systems, further be required by some applicationsfor the image sensors in such an array to accurately slave theirrespective fields of view to a different but still common scenario, orfor the image sensors of such an array to perform an accurate andco-operatively synchronised scan of a common field of regard. Whilstmechanical slaving of image sensors may satisfy some requirementsvirtual image sensors support the fast and accurate positioning of theirfields of view, and allows controlled accurate electronic coordinatedand synchronised scanning between sensors. For all such requirements theimage information from such a multiple image sensor system may need tobe passed to an external processing system.

The use of virtual image sensors is also discussed in more detail laterin the text, these allow the electronic positioning of an image sensor'sfield of view, which need arises primarily because of the necessaryfield of view boresight pointing angle accuracies and slew ratesrequired in image correlation systems, this is unachievable solely frommechanical systems.

Image sensors CCD or equivalent are limited for a particularmagnification to a specific field of view. For image sensors which arecapable of mechanical slaving in azimuth or elevation, the envelope ofthe image sensor's field of view is referred to as the image sensor'sfield of regard.

A virtual image sensor extracts subsets of luminance signals from anarray of appropriately positioned, orientated and synchronised imagesensors, where by combining these luminance subsets with appropriateframe and line sync information the composite video signal so formedallows the real time generation of an image from components of imagesafforded by the array of image sensors, whose adjoining fields of viewsupport the virtual sensor's field of regard equivalent to that of theircombined fields of view, and where the field of view of the virtualimage sensor is equivalent to that of the field of view of one of theimage sensors comprising the array.

Some applications exist where the appropriate positioning of staticimage sensors in such an array covering for example 360 degrees allowssimultaneous multiple fields of view possibilities not achievable from asingle movable image sensor, nor from an array of static image sensorsnot comprising such a virtual image sensor. Further the electronicpositioning of the virtual image sensor's field of view within its fieldof regard can be made faster and more accurately than is possible with amechanical system.

Historically, stable platforms have been achieved through mechanical orelectromechanical functionality. In the context of image sensors suchmethods of stabilization cannot offer the rate and positional accuraciesachievable through electronic stabilization. This aspect offunctionality is also addressed later in some detail. The roll pitch andyaw rates for such a system may be derived from traditional sensors orfrom software tracking of pattern thread motion possible with imagesensors.

Such a sub system is capable of real time operation within the contextof the frame rate of such an image sensor where the data latency of thestabilized data is one frame period. This sub system depends on theability to store image data, contained in the composite video signalfrom an image sensor, in a memory where the address for each pixel ofinformation is generated at the maximum bandwidth frequency of the imagesensor and is corrected in real time to account for roll, pitch and yawdisplacements of the image sensor.

Such a sub system is capable of image derotation traditionally performedby mirrors, but is also intended for example to allow for distributedimage sensor systems in which some components may be cantilevered outand where such components may be subject to local vibrational andbending moment effects of the structure as well as the motion of theentire system. In particular the sub system provides compatiblefunctionality with the topography processors, though it necessitates acoordinated integration of the inverse functionality (as described inthis particular sub system description) across the multiple associatedcomponent level CCD or equivalent sensors comprising a virtual imagesensor, if the frame period data latency is to be avoided.

Image sensors CCD or equivalent, typically with operating bandwidths of5 mhz or greater, generate on a continuous basis considerable volumes ofdata. Analogue to digital converters are becoming increasingly fastermaking real time input of data from such devices, used as front endsensors, to computer processing systems a reality. However the softwaretask of real time data reduction to extract important image patternthread information, particularly when the images relate to a dynamicscenario, presents a considerable processing load to any computer. Manyapplications could benefit from hardware data reduction techniques whichimproves the image pattern thread information to data ratio of the inputfrom such image sensors to computer systems. A more detailedconsideration of such functionality employed in the topography processorsystem, is also addressed later in the text.

The image pattern thread information contained in the composite videosignal of an image sensor is generally contained in the upper region ofthe image sensor bandwidth, and spectral analysis or equivalentprocessing of this signal yields the binary event information ofelements of image pattern threads.

The position within a CCD raster scan, that is within the field of viewof the image sensor, of the occurrence of such binary events is alsodeterminable. Both the binary event information and event identities maybe input to a computer system, the binary event data as an image mappedarray, whilst the binary event identification lends itself to datacompression techniques allowing partitioned (on the basis of line scanand attribute) lists of data to be formed further expediting thesubsequent analysis of such data by other processing systems. Doublebuffering of memories used to pass such data to an external processingsystem allows, on a frame basis, a continuous throughput of data.

Visibility of the output of the image pattern thread processor ispossible by the display of a composite video signal synthesised by thereal time combination of the binary event signal with current strippedframe and line sync information from the image sensor.

The present invention is primarily as defined in the claims, but alsorelates to the problems and solutions discussed above.

According to the current invention, in its first aspect there isprovided a topography processor system comprising a phased image sensorarray, at least one processor arranged to perform range decompression ofimaged detail, and a means for allowing non interruptive graphic macrodiagnosis based on a graphic display of the frame rate macroscopicbehaviour of internal transfer functions of the system to permit visual,augmented visual or automatic visual determination of the system'sintegrity.

The present invention provides in a further aspect a real time genericpattern derived topography processor comprising, three or more imagesensors, CCD or equivalent, of defined separation, orientation andsynchronisation organised as a phased image sensor array where theirfields of view share a common scenario, and one or more processorscapable of processing the image sensors composite video signals toidentify and associate binary elements of generic patterns between allimage sensors in the phased image sensor array thereby resolving thesystem relative spacial position of topographical detail implicit insuch correlated generic patterns.

The present invention provides in a yet further aspect an iterativesubset pattern derived topography processor comprising, two similarimage sensors, CCD or equivalent, of defined position orientation andrelative frame and line synchronisation whose fields of view share acommon scenario, and a processor or processors capable of the iterativesynthesis of combinations of, and real time comparison of, subsetpatterns between sets of such subset patterns derived from the two imagesensors composite video signals, where the multiple correlation of suchsubset patterns members between image sensors allows the system relativespacial position of their associated elements of topographical detail tobe fixed in real time from apriori knowledge.

The present invention provides in a still further aspect a cooperativelyslavable phased virtual image sensor array comprising a number ofequivalent virtual image sensors, logically and physically positionedand orientated, and where the boresights of their respective fields ofview are parallel and such that they generate images of a commonscenario from their different perspectives, and where common imagedetail is registered in corresponding line scans for each such virtualimage sensor, and where the frame and line sync generation for each suchvirtual image sensor is controlled such that a time coincidence existsbetween the characteristics of these signals, and where each suchvirtual image sensor's field of view may be slaved or scan in acontrolled coordinated and synchronised fashion such as to preserve thediffering perspective views of a common scenario within their sharedfield of regard, and where such image information may be passed to anexternal processor system

The present invention provides in a still yet further aspect a virtualimage sensor comprising a number of similar image sensors organised inan array, where the logical and physical position and orientation ofeach such image sensor in the array is such that their individual fieldsof view may be considered collectively to cover a continuous scenariocomprising the individual images from each image sensor, and where it ispossible to generate in real time a virtual image sensor image fromcomponents of images from one or more adjoining image sensors in thearray such that the field of view of the virtual image sensor isequivalent to the field of view of any image sensor in the array andwhere the field of regard of the virtual image sensor comprises theindividual fields of view of the image sensors in the array.

The present invention provides in a still yet further aspect anelectronically stabilized image sensor comprising a memory bufferingsystem which permits three axis positional correction of image sensorimage data in real time where such data is stored address corrected as afunction of the image sensor's motion, achieved through cascaded addresstransformation techniques, in memories during one frame period such thatduring the subsequent frame period the image data may be readsequentially to provide a stable platform equivalent image sensor outputwhere double buffering of the memories used allows for continuousthroughput of image data.

The present invention provides in a still yet further aspect an imagepattern thread processor capable of the real time extraction ofimportant image pattern thread information from the composite videosignal of an image sensor, CCD or equivalent, comprising a luminancesignal spectral analyser or equivalent capable of identifying imagepattern outline and relief contour detail derived from luminance signalfrequency excursions transiting through a preset upper frequency limitor falling below a preset lower frequency limit where such eventsgenerate in real time a binary event signal and an associated identityfor each such event and where such information may be passed to afurther processing or display system.

BRIEF DESCRIPTION OF THE DRAWINGS

A specific embodiment of the invention will now be described by way ofexample with reference to the accompanying drawings in which:—

FIG. 1 shows a simplified combined luminance upper and lower frequencyexcursion event image with time slice XX phased image sensor arrayvector output for lower limit frequency excursions events.

FIG. 2 shows a simplified combined luminance upper and lower frequencyexcursion event image showing for time slice XX phased image sensorarray vector output for upper limit frequency excursions events.

FIG. 3 shows phased image sensor array comprising three virtual imagesensors and wide angle operators image sensor.

FIG. 4 shows simplified schematic of information flow in processing ofsingle line scan of vectors of one attribute between three imagesensors.

FIG. 5 shows system block diagram of phased image sensor array input togeneric pattern derived topography processor.

FIG. 6 shows representation of image sensor, line scan, and vectorattribute partitioned stacks.

FIG. 7 shows input data and write control stack address pointers forimage sensor, line scan, and vector attribute.

FIG. 8 shows system block diagram of organisation for write and read ofone partitioned stack.

FIG. 9 shows system block diagram of stack read control address pointersfor set of line scan stacks holding same vector attribute.

FIG. 10 shows system block diagram for read read write “RRnW”organisation for processing of one set of partitioned stacks holdingsame vector attributes.

FIG. 11 shows system block diagram for combined output from two sets ofline scan vector pair intercept buffers.

FIG. 12 shows system block diagram for clear of two sets of line scanvector pair intercept buffers.

FIG. 13 shows timing considerations for an element of topographicaldetail within the common field of view of the virtual image sensors V1V2 V3.

FIG. 14 shows a schematic organisation of a differential binary eventgenerator.

FIG. 15 shows a schematic organisation of a variable frequency binaryevent simulator and examples of logarithmic displays of vectorintercepts.

FIG. 16 shows closed loop schematic for diagnostic stimulation andmonitoring of the topography processor.

FIG. 17 shows simplified line scan display of vector pair intercepts andscenario three dimensional data base output.

FIG. 18 shows a representation of electro-mechanical relative timeshifting of subset patterns between two image sensors.

FIG. 19 shows a representation of electronic relative time shifting ofsubset patterns between two image sensors.

FIG. 20 shows a block diagram identifying major function areas for asubset pattern topography processing system and signals between them.

FIG. 21 shows an organisation of a subset pattern range processor.

FIG. 22 shows a common scenario viewed by three virtual image sensorscomprising ordered pairs of image sensors of a phased virtual imagesensor array.

FIG. 23 shows a representation of the time distribution of image datafrom an ordered pair of image sensors comprising a virtual image sensor.

FIG. 24 shows a block diagram identifying major functional areas andimportant signals between them for a phased virtual image sensor array

FIG. 25 shows important waveforms used in a phased virtual image sensorarray.

FIG. 26 shows an array of four image sensors comprising a virtual imagesensor where their individual fields of view are aligned to cover acontinuous scenario comprising the separate images of each image sensor.

FIG. 27 shows a representation of the time distribution of informationwithin a frame of composite video from an image sensor.

FIG. 28 shows a representation of the necessary time distribution ofinformation within two frames of composite video from two horizontallyaligned and synchronised image sensors capable of supporting virtualimage sensor subsets in azimuth.

FIG. 29 shows a representation of the necessary time distribution ofinformation within four frames of composite video from four aligned andsynchronised image sensors capable of supporting virtual image sensorsubsets in azimuth and elevation.

FIG. 30 shows a system block diagram identifying important functionalareas and important signals between them capable of supporting a virtualimage sensor.

FIG. 31 shows important signal waveforms used in a virtual image sensorsystem.

FIG. 32 shows a system block diagram for a three axis image sensorstabilisation system, where sensor motion is detected using traditionalsensors, identifying major functional areas and important signalsbetween them.

FIG. 33 shows a system block diagram for a three axis image sensorstabilisation system's address correction functionality, comprising aconfiguration of cascaded ROM's used in this particular example tocorrect normal address identities associated with image data.

FIG. 34 shows a system block diagram for an image sensor stabilisationsystem, where sensor motion is detected through software pattern threadtracking algorithms, identifying major functional areas and signalsbetween them.

FIG. 35 shows a system block diagram of an image pattern threadprocessor identifying functional areas comprising the image patternthread processor and important signal information passed between thesefunctions.

FIG. 36 shows circuitry of luminance differential processor employed inidentifying image pattern thread elements.

FIG. 37 shows detail of the double buffered memories comprising theinter processor data link of an image pattern thread processor subsystem.

FIG. 38 shows images and waveforms associated with the image patternthread processor system.

DETAILED DESCRIPTION OF THE INVENTION

The example of a frame rate generic pattern derived topography processordescribed here is supported by an architecture which permits the realtime three dimensional topographical analysis of a scenario imaged bythree virtual image sensors, CCD or equivalent, organised as a phasedvirtual image sensor array, in which the virtual image sensors fields ofview co-operatively and synchronously scan a field of regard. Thecomposite video generated by the virtual image sensors, of real worlddiffering perspective views of the same scenario is fed into a compositevideo processor comprising image pattern thread processors capable ofthe real time identification and extraction of data elements withparticular and different attributes and which data is alsorepresentative of discrete vectors from each virtual image sensor toelemental detail in the observed scenario. For vectors so identifiedidentities are assigned and these are passed to assigned input stacks ofthe topography processor whose partitioned, parallel and dynamicallyconfigurable architecture and identity combination synthesisers andidentity transform processors are capable of supporting the computationof the existence of common multiple vector intercepts one from eachvirtual image sensor from all combinations of vectors contained withinsets of sets of such vectors having common attributes. The existence ofreal common multiple vector intercepts comprising one vector from eachvirtual image sensor resolves the association between vectors fromdifferent virtual image sensors comprising the phased virtual imagesensor array and the spacial position of topographical detail in theobserved scenario.

The mainline Topography processor system description relates exclusivelyto FIGS. 1-17 of the drawing, subsequent text and figures amplifyaspects of the technologies employed using much of the genericfunctionality and signals described here. With reference to FIG. 1 asimplified (two dimensional) picture of a real world scenario as seen bythe phased image sensor array comprising three virtual image sensors V1,V2, V3. The image represents a combined upper and lower frequencyexcursion event image of the scenario as seen from each virtual imagesensor's perspective. The vectors indicated from each virtual imagesensor position shows, for the time slice XX, the triple vectorintersections for vectors identified from luminance frequency excursionsthrough a lower limit.

With reference to FIG. 2 the same scenario is shown as in FIG. 1 wherefor the same time slice XX the triple vector intercepts are for vectorsidentified from luminance frequency excursions through an upper limit.

With reference to FIG. 3 the phased virtual image sensor array is showncomprising three virtual image sensors V1, V2, V3 each comprising twoimage sensors 1 and 2, 3 and 4, and 5 and 6 respectively. Each pair ofimage sensors comprising a virtual image sensor is positioned, alignedand synchronised such that the boresights of each of their fields ofregard is parallel and that a common scenario is observed between thevirtual image sensors and common image detail of the scenario isregistered in corresponding line scans of each virtual image sensor andwhere further time coincidence exists between the characteristics of theframe and line sync signals of each of the virtual image sensors. A wideangle image sensor 7 with overall view of the scenario allows the fieldof regard of the phased virtual image sensor array to be positioned.

Referring to FIG. 4 which shows a simplified system schematic of asingle vector attribute single line scan processing element comprising anumber of identifiable sub systems including:

A data input sub system 200 comprising three virtual image sensors V1,V2, and V3 organised as a co-operatively slaveable phased virtual imagesensor array which can electronically slave the fields of view of thevirtual image sensors across a common scenario maintaining theirdifferent perspective views. Composite video from the virtual imagesensors is fed to image pattern thread processors IPT1, IPT2, IPT3 whichgenerate sets, per virtual image sensor, of sets, per line scan, of setsof vectors having similar attributes which are stored in physically andlogically partitioned high speed double buffered hardware stacks, wherefor a particular line scan and vector attribute one set 1A, 1B, 2A, 2B,3A, 3B respectively is shown here.

A processing sub system 201 comprising a number of line scan processors(only one shown) organised in parallel and each performing multipleparallel hardware interstack vector identity combination synthesis inthe Combination SYnthesiser CSY 209, and parallel multiple addressidentity transforms in the identity Transform Processors 13TP and 12TP206 which compute the existence of specific vector pair intersectionswhich are stored in dedicated parallel triple buffered ordered vectorpair intercept buffers E,F, G,H, I,J.

An output sub system 202 performing synchronised parallel automaticintercept comparisons between members of sets of vector pairintersections held in the ordered intercept buffers E,F, G,H, I,J.

A control sub system CSS 203 is provided which configures the machine'spartitioned and parallel architecture via sets of three state gatesindicated by the dotted lines and controls the sequencing of theparallel processes which at any time define the state of the machine.

The overall function of such a machine may be described briefly asWRRnWRW process that is as a write 204, read 205, cascaded read 206,write 207, read 208 and write process (performing a “clear” function twoframes out of phase with the write 207 but not shown here).

Elaborating, a series of parallel asynchronous write cycles “W” 204WRRnWRW constituting the data input cycle. A series of synchronousparallel cascaded read cycles terminating in a write “RRnW” 205, 206,207 WRRnWRW form the processing cycle. A series of parallel synchronousread cycles “R” 208 WRRnWRW which generate the system output. A housekeeping cycle comprising a series of clear write cycles “W” WRRnWRW 18not shown but two frames out of phase, for a particular set of sets ofdedicated output buffers, with 207 allows the processes to repeat. Theinput, processing, output and clear cycles operate in parallel,processing data continuously on a frame basis to effect real timeoperation necessitating double buffering of input stacks, paralleloperation of the line scan processors and triple buffering of outputbuffers. The processes may be amplified as follows:

An input cycle 200 where for every frame period vector identity datacharacterising image pattern thread information for the entire framefrom each virtual image sensor in the phased image sensor array V1, V2,and V3 is written asynchronously and in parallel fashion to partitionedhigh speed input stacks. For a single line in frame and specific vectorattribute, the memories 1A, and 1B, 2A, and 2B, 3A, and 3B correspond tothe input stacks for virtual image sensors V1, V2, and V3 respectively.

A processing cycle 201 where during every frame period, data writtenduring the previous frame is read by line scan processors performingsequential cycles of synchronous parallel reads from the partitionedinput stacks generating, in relation to a reference virtual image sensorV1 stack FIG. 41A, or 1B for any particular set of sets of vectoridentities, simultaneous pairs of combinations of vector identitiescomprising one from the reference virtual image sensor set and one fromeach of the other sets of vectors, comprising the same set of sets ofvectors from the other virtual image sensors V2, V3 FIG. 4 2A or 2B, 3Aor 3B respectively. The combinations of vector identities form compoundaddress identities driving a series of parallel cascaded read operationsin the identity Transform processors 13TP and 12TP , whose final outputin each case is an address definition within an ordered output buffer,E,F, or G,H, or I,J one such buffer E,G,I or F,H,J for each set of pairsof sets of vector identities, into which the existence in each case of areal or virtual vector intersection is written. This cycle is repeateduntil all combinations of vector identities between sets of sets of suchvectors has been made. It is also possible as part of the cascaded readsequence for other processor external parameters to interact andthemselves modify address generation, in this particular example such aninteraction is the use (not shown in FIG. 4) of the scanner bearing(FIG. 5 m4) of the boresight of the virtual sensors fields of viewwithin their field of regard. No limit is set on the number of suchcascaded read operations of the identity Transform processors nor on thesize of the address generation at any particular stage of the process,nor on the number of combined operations performed in parallel by theline scan processors. The data sets defining the transforms performed onidentity combinations at the various stages of the cascaded identityTransform processors are generated off line by high level modellingtechniques.

An output cycle 202 where all data processed in the previous frame isnow read sequentially within each ordered output buffer and insynchronous parallel fashion between sets of ordered output buffers(only one set shown in FIG. 4 E&F, G&H, I&J) where the simultaneousexistence, from all members of the same set of buffers (in this exampleE&F or G&H or I&J), of vector pair intersections yields the existence ofa multiple vector intersection, one vector from each virtual imagesensor, whose address identity is the real or virtual spacial positionof an element of topographical detail. This parallel reading of thededicated output buffers resolves the multiple vector intersectionpremise and can simultaneously perform range gating to discriminatevirtual solutions associated with some topographically symmetricalscenarios. This reading of the dedicated output buffers represents theoutput of the system three dimensional data base and is intended tosupport downstream application processing.

The dedicated output buffers are cleared in the frame period followingtheir particular output cycle which necessitates triple buffering ofthese physically and logically partitioned parallel memories.

With reference to FIG. 5 the WRRnWRW 18 processor (representingaccording to system sizing parallel elements of the Control Sub SystemCSS 203 and Combination SYnthesiser CSY 209) input system block diagramshows the individual image sensors 1,2,3,4,5,6 comprising the phasedvirtual image sensor array V1, V2, V3. Image sensor synchronisation isprovided through the functionality of the Frame line mask FLM 8circuitry, Sensor clock synchronisation SCS 11 circuitry, and Fine lineposition FLP 12 circuitry. This is achieved using a lost clock principleto bring the various image sensors to synchronisation. The operator wideangle image sensor IS7 7 also derives its clock drive from the samefunctionality.

The Scanner SCNR 14 allows controlled electronic scanning of their fieldof regard by all virtual image sensors either autonomously or byaccepting an external processor or manual control.

Composite video CV1,2,3,4,5,6 from the individual image sensors1,2,3,4,5,6 respectively in the phased virtual image sensor array isprocessed by the Luminance differential processor LDP 9 circuitry andbinary event data representing upper BEU1,2,3,4,5,6 and lowerBEL1,2,3,4,5,6 frequency excursions through set limits are sent to theScanner SCNR 14 circuitry where virtual imager subsets UVBE1,2,3 andLVBE1,2,3 for upper and lower frequency excursion events appropriate tothe current field of view of the virtual image sensors V1,V2,V3 areextracted and sent to the Stack pointer SP 13 circuitry and WRRnWRW 18processor.

The Stack pointer SP 13 circuitry is driven by binary event data fromeach virtual image sensor, for each attribute, in this example upper andlower frequency excursions UVBE1,2,3 and LVBE1,2,3. It generatessequential stack pointer addresses VISP1U, VISP1L, VISP2U, VISP2L,VISP3U, VISP3L for every physically and logically partitioned virtualimage sensor, line scan, and vector attribute determined stack 114 whichit passes to the write W WRRnWRW 18 processor.

The Video mixer VM 16 and Display D 17 allows operator display of allthe individual image sensor or virtual image sensor, normal video orbinary event data, Scanner SCNR 14 outputs, or generic pattern derivedtopography processor read R WRRnWRW 18 outputs DOP.

Data input to the generic pattern derived topography processor WRRnWRW18 from the Address generation circuitry ADG 10 comprises a framecounter F0, 1, 2 allowing the WRRnWRW 18 processor to schedule doublebuffering of input stacks 114 and triple buffering of the dedicatedoutput vector intercept buffers 116, the time log VI and SI of frequencyexcursion events representative of the azimuth and elevation identitiesof such events respectively within a virtual image sensors field ofview, valid for each virtual image sensor, and the End marker signal EM,representative of the end of each luminance signal, valid for eachvirtual image sensor.

The Scanner's SCNR 14 output to the WRRnWRW 18 processor is m4 whichidentifies the scan position of the virtual image sensors fields of viewboresight within their field of regard, also output are the binary eventsignals UVBE1,2,3 LVBE1,2,3 appropriate to the current scan positionfrom each of the virtual image sensors.

FIG. 6 represents the physically and logically partitioned doublebuffered A&B stacks dedicated to each virtual image sensor, line scanand vector attribute. During any particular frame either the A or B setof stacks will be written to by the “W” write processor logic of theWRRnWRW 18 as a function of the F0 signal from the address generationADG 10 circuitry, allowing the “R” read processor logic of the “WRRnWRW”18 to read the other set. The first subscript denotes virtual imagesensor, the second the line scan SI identity, and the third the vectorattribute, In this example there are only two such attributes those offrequency excursions through an upper “U” and lower “L” set limits.

With reference to FIG. 7 which identifies the elements of thepartitioned stack address pointers and data generation. The vectoridentity VI 20 representative of the azimuth angle in any virtual imagesensor's field of view for any frequency excursion provides common datafor all stacks. The stack identity SI 21 reflects the current line scan,that is line in frame, for all virtual image sensors and is used inconjunction with the vector identity stack pointers VISP1U 22, VISP2U24, VISP3U 26, VISP1L 23, VISP2L 25, VISP3L 27 to generate individualpartitioned stack addresses 28,29,30,31,32,33 for data to be written tothem. The virtual binary event data UVBE1,2,3 and LVBE1,2,3 are used togenerate WE commands for their appropriate stacks.

With reference to FIG. 8 representing the organisation between thewriting and reading of one stack element comprising MA 34 and MB 35.During writing, a stack element is selected according to the virtualimage sensor, line scan SI 21 identity, and particular vector attributeUVBE1,2,3 and LVBE1,2,3. The data written WD is VI 20 position in linescan of a particular event and is common between all virtual imagesensors and stacks for vector identities of different attribute. Thestack element address WA is given by VISP1U 22, VISP2U 24, VISP3U 25,VISP1L 23, VISP2L 25 or VISP3L 27. Write memory control W WRRnWRWgenerates a WE signal for appropriate stacks which is derived from thebinary event data UVBE1,2,3, LVBE1,2,3.

During reading of stack information, cycles of parallel synchronousreads are performed on sets of stacks containing identities, of vectorsof similar attributes, of events which occurred during correspondingline scans of each of the virtual image sensors. Depending on systemsizing a number of such line scan read processes will be performed inparallel by similar line scan processing elements each operating on aset of stacks containing vectors of similar attributes. With referenceto FIG. 9 the reading of stacks is again controlled by pointers, a setof such pointers exist for each parallel line scan processing element ofthe system, which allow selection of a particular set of sensor's stacksaccording to line scan identity and vector attribute. Coordinationbetween such parallel read processes is maintained by pointers, one foreach such process, PSn 120. Within such a line scan read processor twopointers control the actual stack addressing one P1n 49 is the referencevirtual image sensor V1 (n denotes the vector attribute) stack pointer.For every identity read from the reference virtual image sensor V1 stackall identities in the other virtual image sensors V2 and V3 associatedstacks are read, this is controlled in this example by a common pointerP23n 50. Clocking of the reference image sensor stack pointer Pin occurswhen the logical end 53 of either of the other virtual image sensor V2,V3 associated stacks is reached. The reference virtual image sensorstack pointer is reset when the end marker 54 for its own stack data isdetected this also allows each parallel processing element to move on toprocess further sets of stacks, in this example by using the pointer PSn120. For the reading of the other virtual image sensor associated stackstheir pointer P23 are reset by the occurrence of an end marker from anyvirtual image sensor V1, V2, V3 associated stack 53 including thereference stack 54. In this way all combinations of vectors identitiesbetween sets of associated vectors and all sets of such sets are read.The iteration rate of processing solutions depends on the rate at whichthe P23 pointer can be clocked by ITCLK 48 (function of multiple addresstime in identity transform processors). It should be noted that all suchstack reads are performed in parallel and sets of stacks of differentattributes may also be read simultaneously by the use of their owndedicated stack pointers so the actual (vector pair intersection)solution rate for this example of three virtual image sensors and twovector attributes is four times (pair of pair of vector identitycombinations) the iteration rate multiplied by the number of parallelline scan processors.

With reference to FIG. 10 which indicates how data read, as a result ofparallel read “R” WRRnWRW operations, from a particular set of sensorstacks A11U 55, A21U 56. and A31U 57 appears simultaneously on theoutput lines of these particular stacks and passes through the doublebuffering of the three state gates 59 60 61 under control of the “RRnW”76 WRRnWRW sequencing. This first read operation now precipitates aseries of cascaded reads Rn through read only memory ROM using theaddress combinations formed from identities read from the referencevirtual image sensor V1 stack 55 and each of the other virtual imagesensors V2 and V3 stacks 56 and 57 respectively (in response to addressgeneration P1n and P23n).

The combinations of identities taken from the reference virtual imagesensor V1 stack 55 and each of the other image sensor V1 V2 stacks 56and 57 and the identities of the real or virtual vector pairintersections defined by the vectors they represent are known aprioriand implicit in the identity transforms held in ROM. In this example theidentities simultaneously generated at ROM1 62 and ROM3 64 outputsrepresent polar range information of such intersections for thecombination of identities from stacks 55 and 56 for V1 and V2, likewise55 and 57 for V1 and V3 respectively. It will be noted that in parallelROM2 63 address definition has been driven by the reference virtualimage sensor V1 vector identity and the Scanner output m4. The ROM2 63identity transform output represents the vector rotation necessary tosupport the identification of real vector intersections. This rotationidentity output from ROM2 63 now forms address combinations with rangedata identities output from ROM1 62 which are cascaded into ROM4 65 andROM5 66. These ROMs 65 66 generate as outputs from their identitytransforms the identities of the real or virtual vector intersection forthe particular combination of vector identities output from stacks 55and 56. Similarly the identity combination output from ROM2 63 and ROM364 form address combinations into ROM6 67 and ROM7 68 whose identitytransform outputs similarly identifies the real or virtual vectorintersection identities from the combination of vector identities fromstacks 55 and 57. The identity combinations generated by ROM4 65, ROM566 and ROM6 67, ROM7 68 now form the address definitions respectivelyfor the RAM dedicated ordered output buffers 12OBU 74, and 13OBU 75 intowhich the existence of these real or virtual vector intersections arenow simultaneously written by a write pulse generated by the “RRnW”WRRnWRW 76 control. The timing of these write pulses is controlled suchthat the cascaded address generation has become stable following thevarious stages of the cascaded read operations (and this ultimatelycontrols the P23n iteration clock rate ITCLK 48FIG. 9) however, theduration of the complex mathematical computations that have in effectjust taken place are many orders of magnitude less than required forserial computation by software and further the two results producedsimultaneously here represent the output of a single line scan processorelement processing members of one set of sets of members of the samevector attribute, similar line scan processor elements operating inparallel are simultaneously generating their solutions to vector pairintersections from other sets of vector identity stacks. Note that theprecision of the result is a function of the sensitivity of the virtualimage sensors and reflected by the vector identities, which determinethe maximum-size of the address definition passed to ROM1 62, ROM2 63,ROM3 64 however no limit is placed on address definition for any of theROM memories used and by using parallel ROMs, at each stage of theprocess, increasing precision can be achieved in the same effectiveprocess time (multiple address time) as defined by the cascaded ROMidentity transform processor described. The memories used as dedicatedoutput buffers 74 75 are triple buffered allowing continuous processingof data whereby one frame period is used for writing processor output tothe dedicated output buffers, one frame for reading the buffers(scenario data base output), and one frame to subsequently clear thedata base buffers. The triple buffering is achieved in similar fashionto the double buffering of data input, except here three sets (write,read, clear processes) of three state gates are used for address anddata definition for each of the three sets of memories comprising thetriple buffered output memories of which only one set is shown in FIG. 9comprising 69,70,71,72,73.

With reference to FIG. 11 which shows two subsets 74, 75 and 88, 89 ofone set from the sets of triple (not shown) buffered vector pairintercept buffers holding the output from two line scan processors eachhaving processed a set of sets of different vector identities (in thisexample with the attributes upper and lower frequency excursions) forthe same line in frame. During data base output the dedicated vectorpair intercept buffers containing all the identified vector pairintersections written during the previous frame cycle are readsequentially in synchronous parallel fashion. The address definition forthis process is that controlled by the Scanner SCNR 14 SS output throughthe action of the Address generation circuitry ADG 10. From the readingof vector pair intersections from sets of sets of such buffers for aline in frame output of the data base, in this example, 74, 75, and 88,89, their (anded) simultaneous existence at 106 or 107 respectivelyimplies the existence of a unique triple vector intersection where thecurrent output buffer address identifies the three axis special positionof an element of topographical detail.

For the set of sets of intersect buffers here 74, 75 and 88 89, beingread synchronously and in parallel, their outputs are ored forming part(single line scan output) of the sequential output of the data base DOP315. The parallel reading of the vector pair output intersection buffersfor all other pairs of sets and all such sets of sets for all the otherline scan processor output buffers constitutes the output representingthe whole of the observed scenarios topographical three dimensionaldetail, which is intended to comprise the input to a more conventionalprocessing system in support of dependent functions such as for examplescenario navigation. Note again the three state gates 82 83 84 85 90 9192 and 93 effecting address and data triple buffering controlled by the“R” WRRnWRW 81 logic. This example allows for diagnostic purposes simpleoperator sensible displays of a two dimensional subset of this data basefrom any particular set of processors processing sets of sets of vectorswith a common azimuth or elevation attribute by generating a binaryevent signal associated with the existence of a multiple vectorintercept found during the reading of a set of data base buffers. Thisoutput DOP (FIG. 5 and FIG. 11315) is mixed with Scanner SCNR 14produced synthetic frame and line sync SS in the Video mixer 16 toproduce a two dimensional display similar to that of a radar display.With reference to FIG. 17 such a display 263 is shown of a simplifiedimaged scenario representing a horizontal slice through the systemsthree dimensional data base output. Displays 260 and 261 (FIG. 17) showfor the same scenario vector pair intersections formed for the virtualimage sensors V1 V2 and V1 V3 respectively , FIG. 11 signals 348, 349350, 351, are representative of such frame distributed information.

With reference to FIG. 12 all dedicated vector pair intercept buffersnecessitate in this example being cleared in the frame cycle followingdata base output before receiving new intersect data in the followingcycle, this is again effected using addressing from the Addressgeneration circuitry ADG 10. To allow continuous processing requirestriple buffering of the address and data though the action of the threestate buffers 98 99 100 101 102 103 104 and 105 controlled here by theclear control logic which comprises the last write “W” of the WRRnWRW MC18 sequencing logic.

The high level modelling techniques allow for the identification ofimaginary solutions in the definition of the apriori identity transformsfor vector intercepts and other functions employed in the system. Sincethese processors must produce a solution based on their current mix ofinput parameters the identification of anomalous solutions for exampleranging outside the resolution of the system or ranging behind thesystems origin are identifiable and for such cases solutions areproduced which even in cascaded systems will be recognised as imaginaryidentities and given a unique and system recognisable identity to allowexclusion in downstream processes.

Whilst the mathematics of unique multiple vector intercepts generallyworks well in the random real world some situations of symmetry canresult in anomalous multiple vector intercepts, these however can berecognised as short ranging solutions to the unique real world vectorintercepts and can therefore be ignored through range gating (not shown)during the ordered reading of the dedicated vector intercept outputbuffers.

The monitoring of normal video output CV1,2,3,4,5,6,7, (FIG. 5320, 321,322, 323, 324, 325, 326) binary event data BEU1,2,3,4,5,6BEL1,2,3,4,5,6, (FIG. 5 multiple signal lines 327) virtual image sensorvideo CVOP1,2 CVOP3,4 CVOP5,6 (FIG. 5 multiple signal lines 328) andassociated binary event data UVBE1,2,3 LVBE1,2,3 (FIG. 5 multiple signallines 329) have already been earlier addressed, as has the frame rategeneration of the systems three dimensional scenario data base outputDOP (FIG. 5 and FIG. 11315) allowing horizontal or vertical slicesthrough the data base for diagnostic purposes. All of these parametersallow checking of alignment, and calibration of aspects of systemperformance. If necessary such a capability supports the furtherharmonisation of the system either, on installation in its operationalenvironment, or prior to operational use, if for example the distributedimage sensors comprising the array have been disturbed throughmaintenance, or servicing to their host environment.

One aspect of such a topography processing architecture is it's apparentinherent redundancy or resilience if subsystems or parts of subsystemsfail, when the performance of the system may not necessarily appear tosubstantially degrade. This apparent redundancy arises out of thesystem's capability to continuously process data at the image sensor'sframe rate and the inherent noise immunity offered by the multiplevector intersection methodology. In respect of line processing this isdue partially to the double buffering of the physically and logicallypartitioned system input stacks and treble buffering of the system'sdedicated vector pair output buffers. Whilst operationally such featuresenhance particular aspects of the system's performance the need remainsto be able to monitor end to end internal as well as overall systemoperability.

A programmed scrutiny of these lower levels of system processorperformance, without the need to remove boards or use external equipmentis possible because of the frame rate execution of the machine whichallows the integrity of subsystem's and in many cases discretecomponents to be monitored, using the systems prime display, byswitching the required monitor point, or points (described later) withinthe system to the video mixer for combination with appropriate frame andline sync. Generally almost all signal parameters are monitorable eitherin isolation or in combination as described later. This is not in somecases unlike using an oscilloscope to establish a signal's frequencycharacteristics at different points in a television receiver circuitry.The differences being that unlike an oscilloscope the time base is fixed(though generally appropriate) and the Y dimension a unit lineseparation when monitoring clock or counter iteration rates, howeveralso unlike an oscilloscope, macroscopic as apposed to microscopicaspects of the system performance are monitorable over the frame period.Examples are the asynchronous write enable commands to input stacks(FIG. 8336) or vector pair intercept buffers (FIG. 11348,349, 350, 351),or sequential address generation associated with specific vectorattribute processing, or the end to end integrity of multiple addressprocesses (FIG. 10340,341,342,343,344,345,346) in the generation of thevector pair intersections.

Whilst this functionality supports non interruptive graphic diagnosticsmonitoring allowing for example the calibration of write enable pulseduration, or the achievement of minimum multiple address sequenceduration for differing identity processors, the techniques were extendedto allow further diagnostic capabilities allowing aspects of system endto end dynamic performance to be monitored, by the definition andintroduction of standard graphic test formats to establish correctoperation of subsystems. It is difficult with an imager topographyprocessing system located in the room of a building for example to checkthe integrity of the system ranging over say a ten to twenty mile band.To this end three minimal devices were designed to generate simulatedimage vectors representative of the input from each virtual image sensorin the phased virtual image sensor array.

The output from these devices can be introduced to override discreteimager binary event data at the inputs to the scanner, or downstream ofthe scanner at the binary event inputs to the topography processor. Withreference to FIG. 13 showing the fields of view of three virtual imagesensors V1 V2 V3 comprising the phased virtual image sensor array, itcan be seen that for time coincidence of the frame and line synccharacteristics between the virtual image sensors, then for a point oftopographical detail 230 the time within the line scan of the associatedoccurrence of a binary event for virtual image sensor V1 is t1=a+b+c,for virtual image sensor V2 t2=2a+b+2c and for virtual image sensor V3t3=b. With reference to FIG. 14 monostable 221 is triggered by therising edge of the virtual image sensor, scanner produced line sync SS(FIG. 5). The potentiometer forming the X output from a joystick 15 (notshown in FIG. 14) controls the duration “b” 228 of its output, whosefalling edge generates in the chip's other monostable the fixed lengthpulse 225 representative of a binary event for virtual image sensor V3.

The monostable 222 is triggered by the falling edge of waveform “b” 228and generates a pulse of duration “a” 229. The falling edge of thispulse triggers the monostable in the other half of the chip to produce apulse “a′” 230 of similar duration to “a” 229. The duration of the pulse“a” 229 and the chained pulses “a+a′=2a” correspond to the field of viewtime offset associated with the physical separation of the virtual imagesensors V2, V1 and V2, V3 respectively in the phased virtual imagesensor array. The monostable 223 is triggered by the falling edge ofpulse “a” 229 and generates a potentiometer controlled pulse “c” 231whose falling edge triggers the chip's other monostable to produce afixed length binary event for virtual image sensor V1 226. Themonostable 224 is triggered by the falling edge of pulse “a′” 230 andgenerates a potentiometer controlled pulse “2c” 232 twice the durationof monostable's 223 output pulse “c” 231. The falling edge of this pulsetriggers the chip's other monostable to produce a fixed length binaryevent for virtual image sensor V2 227. The potentiometers identified formonostables 223 and 224 are ganged together and form the Y outputs ofthe joystick 15 (not shown in FIG. 14). With reference to FIG. 16 thisminimal circuitry is included in the (FIG. 16) Differential binary eventgenerater DBEG 233, whose switch selectable (through Input switch IS 234) outputs drive the binary event input channels of the topographyprocessor WRRnWRW 18, it is possible to manually and non interruptivelyexercise the topography processor 18 by generating the differentialbinary events between virtual image sensors. The output is effective foreach simulated line scan of the virtual image sensor's, and for thevector attributes binary event input channels into which the simulatoroutputs are connected. Again by the stimulation of only the system'ssufficient control, data and address highways it is possible to monitorthe processor's end to end performance across an envelope of manuallysimulated scenario's topographical detail.

Whilst the system described supports confidence checks of the system'send to end performance the simulation generates only one element oftopographical detail whose position is a function of the essentiallyrandom manual positioning of the joystick 15.

For end to end dynamic processor performance monitoring allowing thefurther possibility of extended graphic macro diagnostic capabilities afurther device whose circuitry is also included in the FIG. 16Differential binary event generator DSEG 233 was designed. The devicewith reference to FIG. 15 comprises a monostable 235 triggered by thefalling edge of the inverted scanner output of virtual image sensor linesync SS which generates a pulse whose duration is “n” 236. The fallingedge of the pulse 236 triggers the chip's other monostable to generate afixed duration binary event signal 237. The falling edge of this pulse237 triggers through the inclusive or gate, the chip's first monostablethereby creating a train of binary events within each line period, whosefrequency is defined by the duration “n” of the pulse 236. The output ofthis device is used to drive simultaneously all binary event inputsUVBE1,2,3 LVBE1,2,3 of the (FIG. 16) topography processor 18 overridingimager inputs via the Input switch IS 234. The effect of thisstimulation is to essentially generate firstly vectors which representtopographical detail outside the range of the system's resolution andtherefore range discrimination but also secondly the device inherentlysymmetrical simulated thread distribution gives rise to all (for givenbinary event distribution frequency) combinations of vector pairsthereby exercising the topography processor 18 over its full operatingenvelope, not only providing visual checks for line scan processduration (for example FIG. 9 monitor 119) characteristics under changingfrequency distribution conditions, but equally positive indications ofthe identity transform processing integrity over an envelope.

For a given frequency of binary event simulation visual, augmentedvisual or automatic visual comparison and checking at all definedmonitoring points of vector intercept calculation is possible againstapriori data as follows. With reference to FIG. 16 showing a part systemschematic of a visual, augmented visual or fully automatic diagnosticstimulation and monitoring configuration. In addition to thedifferential binary event generator DBEG 233 the additionalfunctionality comprises two transform processors 240 and 250 configuredto provide closed loop analysis of the topography processor's 18performance. The first transform processor 240 accepts addressdefinition drive off the address generation ADG 10 functionality andgenerates for this particular system's three virtual image sensors,generic pattern output of an apriori correlatable pattern series. Thesestrobed (not shown) outputs 241 are fed to the topography processor 18binary event inputs (via input switch IS 234), whilst simultaneouslypairs of anticipated vector pair intercepts and scenario threedimensional data base identities 242 are passed to the second transformprocessor 250. For this particular configuration the monitored outputsMnO-Mnn (eg FIG. 11348 349 106 or 106 107 315) 243 switch selected (viamonitor select MS 238) of the topography processor 18, are in fact oneframe period out of phase with the input drive 241. The second transformprocessor 250 accepts the topography processor output 243 and theanticipated system response outputs 242 from the first transformprocessor 240 representing the identities of both vector pair interceptbuffers and scenario data base, and generates for the combination ofthese six (in this particular example) the three outputs 244representing anomalous (differing) outputs 243 from the topographyprocessor 18 and anticipated stimulated output 242. Mixing a switchselected (Output selector OS 239) strobed (not shown) result withscanner frame and line sync SS permits graphic macro display of the set(empty or otherwise) of the actual or anomalous system performancecharacteristics. Clearly in addition to either the display of the visualor augmented visual graphic diagnostics these outputs could equally bebuffered for subsequent analysis (not shown) or in the case of augmentedvisual an automatic failure indication made on the occurrence of a nonempty data set.

This configuration and output monitoring is further capable of otherinternal system transfer function diagnostic monitoring and analysis.For processor internal parameters where the phase difference betweenstimulation and output is less than one frame period then this phasedifference represents a high level modelling parameter in the definitionof the first transform processor's 240 data set. It is furtherinteresting that the second transform processor's 250 transform data setclearly allows processing of multiple or combinations of the topographyprocessor internal parameters in respect for example of simultaneity orotherwise of parameters selected for monitoring. In this fashionmacroscopic sequences of address identities generated by the combinationsynthesiser may be monitored for defined system apriori drive.

A variety of identity transform processors were employed in the originalexperimental equipment, these included cartesian and logarithmic outputdisplays. The particular example cited of a range and bearing processorwas described earlier since its operation in respect of the virtualimage scanning of the field of regard is more efficiently executed inrange and bearing calculations than in cartesian coordinates. Four suchdifferent types of identity transform processors were integrated in theexperimental system's line scan processors where each of the differenttransform processors operating in parallel received simultaneous drivefrom the combination synthesisers (FIG. 4 FIG. 10 not shown). Theoutputs of a particular transform processor set are switch selectedthrough a matrix of three state gates (not shown) to feed theirdedicated vector pair intercept buffers. In practice such an arrangementof output selection supports the selection and use of differenttopographical range scales.

The dynamic envelope performance of all of these transform processesverified through their high level (production) modelling of theirembedded data sets supports, in conjunction with the variable frequencypulse generator 235, system integration and configuration integritychecks. Referring again to FIG. 15 various graphic macro diagnosticdisplays of the system are indicated where logarithmic range transformprocessors are employed. Whilst the normal cartesian or range andbearing scenario three dimensional data base output represents imagessimilar in many ways to those generated by a radar (FIG. 17), theattraction of a logarithmic display particularly in the context of noninterruptible graphic macro diagnostics, is the linearly spaced matrixrepresenting possible vector intercepts display FIG. 15210. The X axisis representative of V1 field of view angle (left to right) whilst the Yaxis represents possible range differentiated V2, V1 vector intercepts(range decreasing top to bottom). A point displayed represents a vectorpair intersection between vectors of a particular attribute (for exampleFIG. 11348, 349, 350. 351) and line scan between, in the case of 210,the two virtual image sensors V1, V2. Display 211 shows the situationfor V1 and V3 where the X axis is V1 field of view angle (left to right)and the Y axis representing possible V1, V3 range differentiated vectorintercepts (range decreasing top to bottom) again points represent thepresence of vector pair intercepts between these two virtual imagesensors V1, V3. Display 212 shows the multiple vector intercepts FIG.11315 for vector pair intercepts common between the three virtual imagesensors V1 V2 V3, in other words the information representing a linescan slice output of the scenario three dimensional data base (in thisformat) albeit deliberately not showing the short range discrimination,since this format is being employed to dynamically and diagnosticallyexercise the system. Displays 210, 211, 212 therefore indicate for thesimulator drive 237 operating at a given frequency of binary eventgeneration some of what may be many thousands of vector interceptscalculated at the frame rate, for a particular line scan, betweenvirtual image sensors in the phased virtual image sensor array.

Scanning the virtual image sensor's field of view offers the possibilityof a practical wide field of regard, and electronic scanning of thevirtual image sensors field of view across its field of regard allowsfast and accurate positioning of its associated boresight offeringpossibilities by effecting a rotation of the field of view to improverange discrimination particularly at extreme ranges. The displayexamples FIG. 15210-218 inclusive are representative of a system inwhich the field of regard of each virtual image sensor is twice that ofthe field of view of the discrete image sensors forming these particularvirtual image sensors. In this example to maximize the azimuth displayresolution of the system output, the identity transform processorsprocess combinations of vectors from the three virtual image sensors inconjunction with the scanner field of view boresight position, and onlyrecognise combinations of vectors capable of generating multipleintercepts in a field of view centred at the middle of the field ofregard of the scanner. Displays 210, 211, 212 show the stimulated systemoutput for the scanners field of view boresight position being at thecentre of the scanner's field of regard. Display 213 shows for virtualimage sensors V1 and V2 vector pair intersections where the scannerfield of view boresight is positioned an eighth of a field of regardleft of the field of regard centre. Similarly display 214 shows asimilar situation for V1 and V3 vector pair intersections while display215 shows the multiple vector intersections between V1 V2 and V3 withthe scanner in the same position. Displays 216, 217, 218 show a similarscenario with the scanner field of view boresight positioned an eighthof a field of view right of the scanner field of regard centre. Itshould be remembered that the simulator output 237, generated as afunction of virtual image sensor line sync output SS from the scanner,is introduced downstream of the scanner at the topography processorsbinary event input channels, thus the simulation represents (for thisparticular diagnostic example to allow dynamic exercising of allmultiple address combinations) a scenario that is moving with thescanners rotation, as opposed to the normal system operation ofstabilised topographical detail displays (unless of course scenariodetail is moving with respect to the system). It is therefore not onlypossible to dynamically examine the system's performance envelope by thesystem's graphic macro diagnostic response in respect of line scanprocessing, but by permitting the scanner to free run the processor'sdynamic end to end performance is also determinable using visual,augmented visual or automatic visual means.

Use of discrete, or combinations of the techniques described, supportsdefinition of a comprehensive range of visual, augmented visual, orautomatic visual graphic macro diagnostic tests, which allow systemintegrity GO/NOGO decisions to be reached non interruptively for such atopography processor system.

With reference exclusively to FIGS. 18-21 a description of an iterativesubset pattern derived topography processor is included here, whichdespite its iterative operation introduces another variation of a rangeprocessor operating at an image sensor's data rate and capable duringsynchronous modes of operation of supporting graphic macro analysis.This example comprises two image sensors with known separation andorientation whose fields of view share a common scenario and wherecommon image detail is contrived to exist in corresponding line scans ofthe two image sensors. For pairs of sets of luminance signal amplitudeand luminance signal time differential elements generated in real timefrom both image sensors the partitioning of such unassociated elementsso as to define subset patterns is made in sympathy with the binaryevent outputs of an image pattern thread processor identifying theoccurrence, for each image sensor, of frequency excursions of theluminance signal through preset upper and lower limits. For such sets ofpairs of subset patterns so identified comprising elements of amplitudeor (exclusive) time differentials of the luminance signal between suchbinary events, combinations of time shifted pairs of pairs of subsetpatterns within corresponding line scans of the two image sensors may beiteratively synthesised by the software of an external processor, orusing electromechanical means of relative motion between the imagesensors fields of view, or by dynamic modification of the image sensorsrelative frame and line sync separation. The real time comparison of themembers of potentially associated (same line scan, same start criterionand attribute) subset patterns between image sensors identifies thosemembers of pairs of pairs of subset patterns where the simultaneousequality of amplitude members and equality of time differential membersto a given precision implies unique vectors from such members with anintercept at an associated element of topographical detail. Aprioriinformation of possible vector intercepts implicit in the geometry ofthe normal relative azimuth position of corresponding subset patternelements between image sensors allows the system relative slant rangeto, and hence the system boresight relative height of such membersassociated elements of topographical detail to be computed in real time.

The software processing of subset patterns necessitates the passing ofluminance signal information partitioned by the outputs from an imagepattern thread processor identifying luminance frequency excursionsthrough upper and lower preset frequency limits for both image sensorover an interprocessor link (not shown).

With reference in particular to FIG. 18 showing a representation ofelectro-mechanical relative time shifting of subset patterns. Two imagesensors IS1 A1 and IS2 A2 of known separation and orientation are shownwhere the boresights of their respective fields of view are parallel andtime coincidence exists between their frame and line synccharacteristics regard a common scenario such that common image detailis contrived to exist in corresponding line scans of both image sensors.A subset pattern A3 is registered at the different luminance signalstart relative times A4 and A5 in corresponding line scans of the imagesensors IS1 A1 and IS2 A2 respectively. The luminance signals for theline scan in which the subset pattern A3 exists are shown A6 and A7 forthe image sensors IS1 A1 and IS2 A2 respectively. An anticlockwiserotation of the right hand Image sensor relative to the lefthand imagesensor in respect of the line scan plane by an angle theta A8 will bringthe subset pattern A3 start time A10 relative to the luminance begintime of image sensor IS2 A2 into time correspondence with the subsetpattern start time A9 relative to the luminance begin time of imagesensor IS1 A1 allowing real time hardware comparison of pairs (differentattributes) of pairs (same attributes) of subset patterns between imagesensors. It should be noted that the subset pattern A10 of IS2 A2 is ofshorter duration (because of the rotation) than the subset pattern A9for image sensor IS1 A1. The rotation angle theta A8, and the separationof the image sensors, and the azimuth angular position of image sensorIS1's A1 subset pattern A3 (function of A4) allows the system relativeslant range to be calculated from apriori knowledge, equally frominformation of the subset pattern's A3 elevation with respect to theimage sensors field of view boresight (function of line in frame) thesystem boresight relative height of the subset pattern A3 associatedelement of topographical detail is also defined. Control of the electromechanical slaving in this system is possible by the functionalitydescribed later for the Shifted sync SSY A20 circuitry where use of theoutput parameter RS would control the relative slaving of one imagesensor's field of view in relation to the other. It should however benoted that the output L1′ should (for this electro mechanical system)represent an unmodified image sensor IS1 A1 frame and line sync L1signal (not shown).

With reference to FIG. 19 showing the same image sensor and scenario asoutlined above but for a system where electronic time shifting of subsetpatterns is employed. The luminance signals for the line scan in whichthe subset pattern A3 exists are shown A6 and A7 for the image sensorsIS1 A1 and IS2 A2 respectively. An inter image sensor relative frame andline sync separation of the image sensors IS1 A1 and IS2 A2representative of a relative time advance of image sensor's IS1 A1 syncin relation to image sensor IS2 A2 by A13 brings time correspondencebetween subset pattern A3 start times A9 for image sensor IS1 A1 and thecorresponding subset pattern A3 start time A12 for image sensor IS2 A2allowing real time hardware comparison of the subset patterns. It shouldbe noted that the duration of subset pattern A3 remains constant. Theapparent field of view rotation of image sensor IS2 A2 in relation toimage sensor IS1 A1 associated with the relative frame and line syncseparation A13 together with the IS1 A1 boresight relative azimuthposition of the subset pattern A3 (function of subset pattern A3 starttime A9) and image sensor separation allows system relative slant rangeto be calculated from apriori knowledge, equally from information of thesubset pattern's elevation with respect to the image sensor's field ofview boresight (function of line in frame) the system boresight relativeheight of the subset pattern A3 associated element of topographicaldetail is also defined.

With reference to FIG. 20 two image sensors IS1 A1 and IS2 A2 CCD orequivalent are positioned with known separation and orientation suchthat their fields of view share a common scenario and allow differentperspective views of the same scene. The physical alignment of the imagesensors is such that common image detail is registered in correspondingline scans of each image sensor. The synchronisation of the imagesensors in respect of their Frame and line sync generation isdynamically controlled from the Sensor clock synchronisation SCS A14circuitry such that the relative synchronisation separation betweenimage sensors lies within one line scans luminance period of each other.The default condition is time coincidence of the image sensors Frame andline sync generation L1 and L2 respectively. Composite video CV1 and CV2from image sensors IS1 A1 and IS2 A2 respectively is fed to the Frameline mask FLM A15 circuitry, Luminance differential processor LDP A17circuitry, Analogue to digital converter ADC A18 circuitry, and theVideo mixer VM A29 circuitry.

The purpose of the Frame line mask FLM A15 circuitry is to strip fromeach of the composite video signals CV1, CV2 the Frame and line syncinformation L1, L2 for image sensors IS1 A1 and IS2 A2 respectively.From each of the stripped sync signals L1, L2 two further signals aregenerated for each image sensor, a Frame begin signal F1, F2 and aluminance mask signal M1, M2 respectively. The stripped Frame and linesync signal L1 is sent to the Shifted sync SSY A20 circuitry. Thestripped Frame and line sync signal L2 is sent to the Fine line positionFLP A16 circuitry. The Frame begin signals F1, F2 are passed to theSensor clock synchronisation SCS A14 circuitry. The Frame begin F1,Frame and line sync L1, and Luminance mask M1 signals are sent to theAddress generation ADG A19 circuitry. Mask information M1, and M2 aresent to the Luminance differential processor LDP A17 circuitry andAnalogue to Digital converter ADC A18 circuitry.

The purpose of the Sensor clock synchronisation SCS A14 circuitry is tocontrol clock drive CL1, CL2 to the image sensors IS1 A1 and IS2 A2 soas to establish, maintain, and in conjunction with the Fine lineposition FLP A16 circuitry and Shifted sync SSY A20 circuitrydynamically modify the relative synchronisation of the two imagesensors. Initially the Sensor clock synchronisation SCS A14 circuitryestablishes synchronisation of IS1 A1 and IS2 A2 such that a timecoincidence exists between Frame and line sync generation for the twoimage sensors. This is achieved by monitoring the Frame begin signalsF1, F2 for simultaneity and stopping the clock drive to either of theimage sensors so as to achieve this, when the signal Frame lock FLK isset. The Frame lock signal FLK allows the Fine line position FLP A16circuitry to finely adjust the relative synchronisation of the imagesensors on the basis of individual line scans as determined by the L1′and L2 signals when fine adjustment of the IS1 A1, IS2 A2 clock drives.CL1 and CL2 respectively through the action of the line lock signalLLK1, LLK2 is possible. Initially the Shifted sync SSY A20 circuitryoutput L1′ represents the unmodified Frame and line sync signal L1.

The purpose of the Shifted sync SSY A20 circuitry is to modify theapparent relative timing of the image sensor IS1 A1 Frame and line syncsignal L1, by generating a time shifted sync signal L1′ which is sent tothe Fine line position FLP 16 circuitry, this has the effect ofcontrolling the relative separation RS of the actual L1, L2 Frame andline sync generation by image sensors IS1 A1 and IS2 A2 respectively.The signal RS is sent to the Range processor RP A23 and the Read writememory control RWMC A27 circuitry. The Shifted sync SSY A20 circuitryoperates in two main modes, either under the control of the Externalcomputer system ExCs A28 or in an autonomous mode. When under thecontrol of ExCs A28 processor the magnitude of the time shift in L1′(relative time advance of L1 in respect of L2) characteristics relativeto L1 is defined by the Time shift and control TSC parameters passedacross the inter processor link IPL A31.

This mode allow s the ExCs A28 processor to control range discriminationover a specific and programmable bond of ranges. In the autonomous modethe magnitude of the L1 signal characteristics time advance with respectto L2 is controlled by a sweep generator which allows the relative Frameand line synchronisation of IS1 A1 and IS2 A2 to iteratively change stepwise on an inter frame basis from time coincidence to a maximum of oneline scans luminance period relative time advance of L1 in respect ofL2. The iterative change of the relative synchronisation L1, L2 may bemodified by the action of the Range lock signal RL.

The Address generation ADG A19 circuitry accepts clock drive CL1 fromthe Sensor clock synchronisation SCS A14 circuitry and the signals F1,L1, and M1. The clock CL1 is gated by the mask signal M1 and drives aseries of cascaded binary counters parts of which are reset by thecharacteristics derived from the F1, and L1 signals, this allows addressgeneration for each resolution period of the image sensor IS1 A1 andcorresponds to the image sensor IS1 A1 current raster scan positiondefining line in frame SI (representative of elevation angle within theimage sensor's field of view) and position in line VI (representative ofazimuth angle within the image sensor's field of view). The Addressgeneration circuitry ADG A19 also generates a signal F0 which countsframes of data and allows through this signal the double buffering ofdata passed across the Inter processor link IPL A31. The signals SI andVI are sent to the Toy stick comparator JSC A22 circuitry and Read writememory control RWMC A27 circuitry.

The Joy stick A26 generates X Y demands which are converted in theanalogue to digital converter XYC A25 circuitry to digital values whichare passed to the Joy stick comparator JSC A22 circuitry and Read writememory control RWMC A27 circuitries. The digitised X Y demands arecompared in the joy stick comparator JSC A22 circuitry with therespective VI (position in line) and SI (line in frame) address identitycomponents for image sensor IS1's A1 current raster scan position,equality of these operands sets the signal EQ which is sent to theSubset pattern comparator SPC A24 circuitry. The current X Y demands arepassed via the Read write memory control RWMC A27 across the Interprocessor link IPL A31 to the External computer system ExCs A28. Undercontrol of the External computer system ExCs A28 image mapped overlaysymbology OS may be generated and passed via the Inter processor linkIPL A31 to the RWMC A27 circuitry which in sympathy with the SI and VIraster scan identities can pass the overlay symbology to the Video mixerA29 circuitry for superposition on image sensor IS1's A1 composite videosignal to form the composite video CV3 for display on Display D A30.This overlay information supports the operator's joystick controlfeedback loop.

The Luminance differential processor LDP A17 circuitry accepts compositevideo CV1, CV2 from the image sensors IS1 A1, IS2 A2 respectively andLuminance mask signals M1, M2 from the Frame line mask FLM A15circuitry. The purpose of the Luminance differential LDP A17 circuitryis to identify image pattern thread data, that is pattern outline andrelief contour detail, according to the criteria of luminance signalfrequency excursions through preset upper or lower frequency limitswhich generate the binary event signals BEU1, BEL1 and BEU2, BEL2representing the binary events of frequency excursions through the upperand lower frequency set limits for the image sensors IS1 A1 and IS2 A2respectively. These four signals are sent to the Event time coincidenceETC A21 circuitry.

The purpose of the Event time coincidence ETC A21 circuitry is toestablish when similar binary events (that is for upper or (exclusive)lower frequency excursions events) occur simultaneously from both imagesensors when one of the signals BEPU and BEPL is set, representative ofinter image sensor binary event punctuation for upper and lowerfrequency excursions events respectively. These signals are reset at thestart of every image sensor IS1's A1 luminance period as a function ofthe Mask signal M1, or by the subsequent occurrence of each other (thatis a subsequent simultaneity of the signals BEU1,BEU2 or BEL1,BEL2 beingset when additionally either BEPU or BEPL respectively would be setagain), or the isolated occurrence of either of the signals BEU1, BEU2,BEL1, BEL2. The signals BEPU and BEPL are passed to the Subset patterncomparator SPC A24 circuitry.

The Analogue to digital converter ADC A18 circuitry accepts thecomposite video signals CV1 and CV2 from image sensors IS1 A1 and IS2 A2respectively, the mask signals M1, M2 from the Frame line mask FLM A15circuitry and the clock signal CL1 from the Sensor clock synchronisationSCS A14 circuitry. The Analogue to digital converter ADC A18 circuitrygenerates the digital amplitude and time differential of each of theluminance signals CV1 and CV2 and identifies (when both masks M1 and M2are set) equality (to given precision) of pairs of elements of luminanceamplitude and equality (to given precision) of pairs of elements ofluminance time differential signals between the image sensors IS1 A1 andIS2 A2 generating the signals luminance amplitude equality LE andluminance time differential equality DE respectively. The signals LE andDE are passed to the Subset pattern comparator SPC A24 circuitry.

The purpose of the Subset pattern comparator SPC A24 is to establishusing a multiple condition, correlation between pairs of pairs of subsetpattern elements. The Subset pattern comparator SPC A24 circuitryaccepts BEPU, BEPL from the Event time coincidence ETC A18 circuitry,the signals LE and DE from the Analogue to digital converter ADC A18circuitry, and the signal EQ from the Joy stick comparator JSC A22circuitry. When either of the signals BEPU or BEPL are set and both ofthe signals LE or DE are set the implication is that for the currentimage sensor relative sync separation RS and the real time comparison bythe Analogue to digital converter ADC A18 circuitry of time shiftedluminance signal attributes (amplitude and time differential) then forthe pairs of pairs of subset pattern elements between image sensors nowbeing considered a multiple match comprising the same subset patternstart criterion and equality between pairs of luminance amplitudeelements and equality between pairs of luminance time differentialelements has been found. This multiple condition results in the signalPattern match PM being set.

This signal PM is passed to the Read write memory control RWMC A27circuitry. When both pattern match PM is set and the scan equality EQsignal is set then an operator designated feature has been identifiedand ranged, in this mode the Range lock RL signal is set by the subsetpattern comparator SPC A24 circuitry and passed to the Shifted sync SSYA20 circuitry which then stops iterating through the entire image sensorrelative sync separation sequence (entire range sequence) and performstracking of the designated feature by allowing only the minimalperturbations to the relative sync separation between the two imagesensors necessary to effect this.

The Range processor RP A23 circuitry, comprises an identity transformprocessor, which accepts on a continuous basis the signals, currentrelative synchronisation separation RS from the Shifted sync SSY A20circuitry, and the Address generation ADG A19 output of SI and VI linein frame and position in line identities, these allow the real timecomputation of the identities of system relative slant range and systemboresight relative height based on the current mix of parameterssubmitted and which in the presence of the Pattern match signal PM wouldcorrespond with ranging to a real element of topographical detail. Thisdata is passed on a continuous basis to the Read write memory controlRWMC A27 circuitry which allows, in conjunction with the signal PM beingset, entries to be made in the systems, three dimensional scenario database.

With reference to FIG. 21 which shows an organisation of the Rangeprocessor RP A23 comprising Read only memories ROMs supporting identitytransform processes. The purpose of the range processor is to calculate,within every resolution period of the image sensor, from aprioriknowledge of possible vector intercepts between pairs of correlatedsubset pattern elements (one from each image sensor) the system relativeplan range PR, bearing B, and height HT of the associated element oftopographical detail. This information is presented on a continuousbasis to the Read write memory control RWMC A27 which interprets thisdata as real or imaginary on the basis of the Pattern match signal PMbeing set. For the known image sensor separation and orientation, thepartial address component VI and Shifted relative frame and line sync RSpresented to ROMI A32 generates as output a system Slant range SRidentity which is passed as a partial address component to ROM2 A33 andROM4 A35. ROM2 A33 also accepts the partial address component SI andresolves the Slant range SR identity through the SI implicit currentsystem boresight relative elevation angle to produce as output thesystem axis relative height HT to the element of topographical detail.The Slant range SR identity passed as a partial address component toROM4 A35 now combines with the partial address component SI. The outputof ROM4 A35 is the system axis Plan range PR to the element oftopographical detail. ROM3 A34 accepts only the azimuth angle VIparameter as an address definition and generates as output a systembearing to the element of topographical detail. The maximum precision ofthe processor output is a function of the separation and resolution ofthe image sensors reflected in the input parameter precision anddefinition of the data sets defined for the ROMs. The ROMs may be stagewise added to in parallel fashion to gain increased precision of theoutput identities at no loss to the effective multiple address time ofthe processor. The data sets contained in the ROM'S A32-A35 are definedoff line using high level modelling techniques.

The Read write memory control RWMC A27 circuitry accepts continuous planrange bearing and height information output from the Range processor RPA23 circuitry. The subset of this data representing valid elementaltopographical detail ranging information is identified by the Patternmatch PM signal being set when, by using the frame identity signal F0from the Address generation circuitry ADG A19 circuitry entries of thisvalid range information are made into the double buffered memoriescontained within the Read write memory control RWMC A27 circuitrycomprising the systems three dimensional scenario data base allowingaccess by the External computer system ExCs A28 via the Inter processorlink IPL A31.

For an operator designated marking of-image sensor IS1's A1 displayedimagery via the Video mixer VM A29 on Display D A30 the occurrence ofthe Range lock signal RL (from the subset pattern comparator SPC A24circuitry) allows dedicated return of this unique range, bearing andheight information to the External computer system ExCs A28 through thedouble buffered memories via the Inter processor link A31.

In relation to graphic macro diagnostics, this system which is largelyasynchronous does afford a number of possibilities for macro diagnosticanalysis including the anticipated monitoring of image sensor videosignals by the introduction of CV1 and CV2 to the display D A30. Mixingof the image pattern thread processor LDP A17 outputs BEL1, 2 and BELU1,2 with appropriate frame and line sync signals L1 and L2 respectivelyallows the display of image sensor IS1 and IS2 upper and lower frequencyexcursion events. Mixing of IS1 luminance signal or binary event datawith IS2 frame and line sync allows monitoring of the shifted sync SSYA20 functionality. Mixing of the pattern match PM signal with L2 allowsa screen mapping of correlatable range information. Stability of the RLrange lock signal over a frame period is monitorable by mixing of thissignal with either L1 or L2 for display of a composite video signal on DA30. Whilst these are only a few of the obvious examples of the use ofvisual macro diagnostics techniques for this type of system, and whistsuch a system in respect of stereo vision may be in sympathy with our(human) expectation of mapping systems, it suffers from a relativelyslow (iterative) approach to single point ranging. Further in relationto generic pattern derived ranging such ranging has only a limitedprobability of validity since the noise rejection of such a system inrelatively poor.

Referring exclusively to the FIGS. 22-25 a phased virtual image sensorarray is described in greater detail here and utilises some genericfunctionality and signals already introduced in earlier text. Withreference initially to FIG. 22 which shows a horizontally aligned phasedvirtual image sensor array BV1, BV2, and BV3 comprising three orderedpairs of image sensors CCD or equivalent B1 and B2, B3 and B4, B5 and B6respectively. The orientation of the image sensors comprising an orderedpair supports a field of regard comprising the adjoining fields of viewof each image sensor of the pair. Each virtual image sensors ispositioned at known separation from each other in the phased array, inthis example horizontally aligned such that their orientation ensuresthat the boresights of the fields of view of each of the virtual imagesensors are parallel and that the field of view of each virtual imagesensor shares a common scenario with the other virtual image sensors andwhere common image detail is contrived to register in corresponding linescans of the virtual image sensors.

A representation of the time distribution of image information from atypical image sensor FIG. 23 shows visible luminance data B7, frame andline sync, porch and non visible luminance information B8 with a Framebegin time B9. For the ordered pairs of image sensors comprising eachvirtual image sensor the relative image sensor synchronisation isorganised such that the lefthand image sensor of a pair has frame begintime B9 whilst the righthand image sensor of the pair has the relativeframe begin time B12. Visible luminance information for the righthandimage sensor of the pair exists in the rectangle B10, the border areaB11 comprises the frame and line sync, porch and non visible luminanceinformation of this image sensor. A time slice BXBX would comprise linescan information B13 and B14 from the two image sensors comprising anordered pair such that luminance information between them is continuousin time. A subset of luminance information B15 comprising a field ofview equivalent to the field of view of either image sensor in a pairmay be extracted and combined with synthetic frame and line syncinformation B16 to form a frame of composite video signal of a virtualimage sensor with Frame begin time B17. This field of view Bl5 iscapable of being slaved within a field of regard as defined by thecombined fields of view B7 and B10 of each image sensor in the pair, inthis example allowing an azimuth scan.

With particular reference to the system block diagram FIG. 24 compositevideo CV1,2,3,4,5,6 from the image sensors B1,B2,B3,B4,B5,B6respectively is passed to the Frame Line Mask FLM B18 circuitry whichgenerates Frame begin signals F1,2,3,4,5,6 waveform B35, Frame and Linesync signals L1,2,3,4,5,6 waveform B38, and Luminance Mask signalsM1,2,3,4,5,6 waveform B39 for each of the image sensors. Frame beginsignals F1,2,3,4,5,6 are passed to the Sensor Clock Synchronisation SCSB21 circuitry and Frame and line sync L3,4,5,6 are passed to the FineLine Position FLP B22 circuitry. Frame and line sync information L1, L2from image sensors B1 and B2 are normally passed unmodified via theShifted SYnc SSY B60 circuitry as L1′ and L2′ respectively to the Fineline position FLP B22 circuitry. Frame and line sync information L1 isalso passed to the Scanner SCNR B23 circuitry.

The purpose of the Sensor clock synchronisation SCS B21 circuitry is toestablish and maintain the necessary relative Frame and linesynchronisation between the image sensors B1,B2,B3,B4,B5,B6 in thearray.

The Sensor clock synchronisation SCS B21 circuitry achieves this inconjunction with the Fine line position FLP B22 circuitry by controllingthe clock drive to each of the image sensors in the array therebycontrolling each image sensors Frame and line sync generation whichprovides the control feedback loop to the SCS B21 circuitry. In thisparticular example image sensor IS1 B1 is used as a reference sensor,and the relative Frame begin time synchronisation of left and righthandimage sensors of a pair is such as to correspond with that shown in FIG.23 where the Frame begin time for the lefthand image sensor of a pairIS1, IS3, and IS5 is represented by B9, whilst the relative Frame begintime synchronisation for the righthand image sensor of a pair IS2, IS4,and IS6 is represented by B12. The SCS B21 circuitry also generates aFrame window FW signal waveform B36 which corresponds with the framesyncing period this signal is passed to the Scanner SCNR B23 circuitry.

The purpose of the SCaNneR SCNR B23 circuitry is to allow controlledautomatic or manual scanning of each virtual image sensors field ofregard by the virtual image sensors field of view. It comprises a clockwhose pulse rate is variable and during frame sync periods as defined bythe signal FW waveform B36 drives a cascaded count up counter whosemaximum count corresponds with the number of sensor resolution periodsin a luminance period. This counter automatically resets when themaximum count is reached. During any particular frame period this countup register holds a particular and fixed value which represents theScanner's Frame and line sync RS time offset, (image sensor resolutionperiods), relative to the reference sensor's IS1 B1 Frame and line syncgeneration. The output from this counter is loaded for every line scanof a frame into cascaded count down registers by the action of the imagesensor IS2 mask signal M2 waveform B39 going low. A clock also gated bythe Mask M2 and operating at the maximum frequency of the image sensorsbandwidth, counts down the value loaded into the cascaded count downregisters. When the output of all these cascaded count down registersreach zero, elements of frame and line sync information are generated,which over the course of a frame period forms the synthetic Frame andline sync information SS to be combined with subsets of luminancesignals LS1,2, LS3,4, LS5,6 extracted from the image sensors of theordered image sensor pairs B8 and B2, B3 and B4, B5 and B6 respectively,thereby forming for each virtual image sensor a composite video signal.Because the relative Frame and line sync generation between each imagesensor in an ordered pair is the same and because each lefthand imagesensor synchronisation of an ordered pair corresponds with thesynchronisation of image sensor IS1 the synthetic sync information SS sogenerated is appropriate for the combined luminance subsets for allvirtual image sensors formed by each ordered image sensor pair in thephased virtual image sensor array.

Since in this example the image sensors are horizontally aligned virtualimage sensor subsets of luminance signals may only be offset in ahorizontal direction, there being no sensible possibility here to scanthe combined field of views in a vertical plane. To generate thenecessary luminance subset masks, image sensor IS2's 82 Mask M2 risingedge triggers a monostable generating a signal Q whose maximum durationis greater than 0.064 ms. The presence of signal Q forces the selectionof luminance from image sensors IS2 B2, IS4 B4, and IS6 B6 for thevirtual image sensors BV1, BV2, and BV3 respectively. The monostable isreset by the action of the countdown registers producing a zero resultand the presence of signal Q's compliment Q is the mask signal used toselect luminance subsets from image sensors IS1 B1, IS3 B3, and IS5 B5for the virtual image sensors BV1, BV2, and BV3 respectively. Combinedluminance subset signals are selected and combined through analogueswitching gates controlled by the signals Q and its compliment Q andform the Luminance signals VL1, VL2, and VL3 of the virtual imagesensors BV1, BV2, and BV3. Composite video CV7, comprising a particularvirtual image sensors processed luminance subset signal combined withshifted Frame and line sync information SS through the action of theVideo mixer VM B24 may be displayed on Display D B26.

In this particular example an image pattern thread processor comprisingLuminance Differential Processor LDP B19 circuitry, Address generationADG B20 circuitry, and Stack pointer SP B27 circuitry is employed. TheLDP Bl9 circuitry essentially performs spectral analysis of theluminance signal from each image sensor. The binary event signals BE1,2, 3, 4, 5,6 produced by the luminance differential processor from thecomposite video signals CV1,2,3,4,5,6 of the image sensorsB1,B2,B3,B4,B5,B6 respectively are also passed to the Scanner SCNR B23circuitry. The binary nature of these signals allows direct logic gatingwith the subset luminance mask signals Q and its compliment to generatethe virtual image sensor binary event signals VBE1, VBE2, and VBE3 ofeach of the virtual image sensors BV1, BV2, and BV3 respectively. Thesesignals may be similarly combined with the synthetic Frame and line syncinformation SS in the Video mixer VM B24 to form the composite videosignal CV7 for display on Display D B26.

Automatic electronic co-operative scanning rates of the virtual imagesensors across their field of regard are manually variable bymodification of the scanners SCNR B23 variable rate clock frequency.This example allows the output from the integrator count up counter tobe overridden and control and slave position XSC taken from an externalprocessor, simplified here by the use of the digitised output from the Xpotentiometer demand of a Joystick B25. This feature allows synchronousand co-ordinated horizontal scanning by each of the virtual imagesensors fields of view under manual control, across the field of regardformed by the combined fields of view of an image sensor pair. Thesynthetic Frame and line sync SS generated by the Scanner SCNR B23appropriate to any particular position of the slaved virtual imagesensor field of view boresight is passed to the Frame line mask FLM B18circuitry which generates the associated Frame begin signal VF waveformB35, Frame and line sync signal VL waveform B38, and Mask signal VMwaveform B39 appropriate for each of the virtual image sensors in thearray, these signals are passed to the ADdress Generation ADG B20circuitry. The SCNR B23 also generates as output the signal m4, suitablefor passing across an inter process link, which defines the actual scanposition of each virtual image sensor's field of view boresight withinits field of regard common in this example for all virtual imagesensors.

The Address generation circuitry ADG B20 circuitry supports output to anExternal computer system ExCs B29 of image data from the virtual imagesensor array in this example for the particular vector attributesidentified by the LDP B19 circuitry. To achieve this the ADG B20circuitry generates identities representative of the azimuth VI,position in line, and elevation SI, line in frame, within a virtualimage sensors field of view of any possible frequency excursion eventsidentified by the LDP B19 circuitry, both the signals VI and SI arepassed to the Write memory control WMC B28 circuitry. In this particularexample the memory used is physically partitioned according to frame,virtual image sensor, and binary event attribute, but only logicallypartitioned within a frame on the basis of the line scan SI identity.The Address generation ADG B20 circuitry also generates an End marker EMsignal, this signal is passed to both the Write memory control WMC B28circuitry and Stack pointer SP B27 circuitry and is used to terminateline stacks at the end of each virtual image sensors luminance period.The ADG B20 circuitry also generates the signal F0 which identifies oddand even frames and thereby allows the Write memory control WMC B28circuitry and External computer system ExCs B29 double buffering ofmemories M1AB, M2AB, and M3AB used in the inter processor link.

Binary event data output from the Luminance differential processor LDPB19 for each virtual image sensor is passed to the Stack Pointer SP B27circuitry which generates for each of the virtual image sensors apointer VISP1, VISP2, VISP3 within each stack, contained in the memoriesM1AB, M2AB, and M3AB respectively to the particular address to be used,at the next binary event from a particular virtual image sensor, tostore its associated vector identity VI.

The Write memory control WMC B28 combines the information from theAddress generation ADG B20 circuitry and Stack pointer SP B27 circuitryto generate specific addresses WA1=SI+VISP1, WA2=SI+VISP2, andWA3=SI+VISP3 in partitioned memory into which identities WD1=WD2=WD3=VIfor frequency excursions events from a particular virtual image sensorand line scan and attribute are written. The binary event signals from aparticular virtual image sensor in conjunction with the signal F0 areused to generate the WE signals WE1A, WE1B, WE2A, WE2B, WE3A, and WE3Bfor the currently available memory associated with a specific imagesensor. F0 allows the double buffering of memory where address and datalines are buffered for writing as well as reading by the use of sets ofthree state gates for the A and B memories involved, allowing anExternal computer system ExCs B29 continuous processing of frameorganised data.

For the generic pattern derived topography processor the functionalitydescribed above is sufficient, however in the context of the subsetpattern topography processor the functionality of the shifted sync SSYcircuitry is important and follows. The purpose of the Shifted sync SSYB60 circuitry is to override the co-operative synchronous scan of thevirtual image sensors and permit relative to the reference virtual imagesensor an asynchronous scan (ie non parallel field of view boresightsand non time coincidence between virtual image sensor frame and linesync signals) by the other virtual image sensors. This feature allowsthe fields of view of the other virtual image sensors to effectivelyscan the field of view of the reference virtual image sensor. This isachieved by modifying the apparent relative timing of the virtual imagesensor V1's Frame and line sync signal by generating a time shifted syncsignal L1′ and L2′ from the signals L1 and L2 respectively which aresent to the Fine line position FLP B22 circuitry, this has the effect ofcontrolling the relative separation of the Frame and line syncgeneration by virtual image sensors BV2 and BV3. The Shifted sync SSYB60 circuitry operates in two main modes, either under the control ofthe External computer system ExCs B29 or in autonomous mode. When underthe control of ExCs B29 the magnitude of the time shift in L1′ and L2′characteristics relative to L1 and L2 respectively and thereby BV1relative to BV2 and BV3 is defined by the Time shift and control TSCparameters. In the autonomous mode the magnitude of the BV1 frame andline sync characteristics delay with respect to BV2 and BV3 iscontrolled by a sweep generator which allows the relative Frame and linesync of virtual image sensors BV2 and BV3 in respect of the referencevirtual image sensor V1 to iteratively change step wise for each newframe period from time coincidence to a maximum of one line scanluminance period.

With reference exclusively to FIGS. 26-31 a more detailed description ofvirtual image sensor functionality is included here which employsgeneric functionality and signals already introduced in earlier text. Avirtual image sensor or sensors comprises a number of image sensors ofequal magnification organised in an array, where the logical andphysical position and orientation of each such image sensor in the arrayis such that the individual fields of view of such image sensors may beconsidered collectively to cover a continuous scenario comprising theindividual images from each image sensor. It is possible, by suitablycontrolling the frame and line synchronisation signal generation of eachsuch image sensor, that luminance information continuous in time may beconsidered to exist between corresponding line scans of adjacent imagesensor image boundaries, such that one or more subsets of luminanceinformation may be taken from one or more of the adjacent image sensor'scomposite video signals in real time and combined with appropriatesynthetic frame and line sync information so as to generate thecomposite video output of a virtual image sensor or sensors. The fieldof view of such a virtual image sensor or sensors is equal to the fieldof view of any image sensor in the array. It is possible under manual orautomatic control, by modifying in each case both the subsets ofluminance signals extracted from individual image sensors in the array,and the generation of its associated synthetic frame and line syncinformation, to accurately slave each of the virtual image sensor orsensors field or fields of view to particular or different points in, orscanning with each of the virtual image sensor's field of view a fieldof regard comprising the individual fields of view of the image sensorsin the array.

With particular reference to FIG. 26 an image sensor's field of view maybe considered of pyramid shape with rectangular cross-section extendingfrom the image sensor such that in this example an array of four suchsimilar image sensors C1,C2,C3,C4 are organised as an array and soaligned as to collectively view a continuous scenario comprising thecomponent images of each the image sensors.

With reference to FIG. 27 the time distribution of information containedin the composite video signal of an image sensor is such that displayvisible luminance is contained in the central rectangle C5 and sync,porch and non visible luminance information is contained in the borderarea C6. Within a frame of data waveform C31 time begins, (Frame begintime waveform C33), at the extreme top left corner C7 and increases leftto right in successive horizontal lines from top to bottom. A particulartime slice CXCX within a frame of data is shown C8. For a UK system thetime from the extreme top left C7 to extreme bottom right C35 is 20 msand the duration of an individual line scans is 0.064 ms.

With reference to FIG. 28 a representation of the necessary timedistribution of information between two image sensors horizontallyaligned and capable of supporting a virtual image sensor where frames ofdata would comprise luminance subsets in azimuth C16 is shownnecessitating the relative time offset of Frame begin time C11 of therighthand image sensor of the pair, to the Frame begin time C7 of thelefthand image sensor of the pair. It can be seen that the luminanceregions C5 and C9 of the two image sensors may be considered line-wiseto be continuous in time and a time slice at CYCY would comprise theline scans C12 & C13 of the left and right hand image sensorsrespectively.

Similarly for the four such image sensors C1 C2 C3 C4 positioned andaligned according to FIG. 26 then the necessary time distribution, withreference to FIG. 29, of information to support virtual image sensorframes of data comprising subsets of luminance information in bothazimuth and elevation C23 requires that the relative Frame begin timesfor the image sensors C1,C2,C3,C4 is given by the Frame begin timesC7,C11,Cl9 and C22 respectively. In this particular example the Framebegin time C11 of image sensor C2 is offset from the Frame begin time C7of image sensor C1 by the same amount as the offset of the Frame begintime C22 for image sensor C4 in respect of the Frame begin time C19 forimage sensor C3, further the offset between Frame begin times C7 and C19for image sensors C1 and C3 respectively is the same as the offsetbetween Frame begin times C11 and C22 that for image sensors C2 and C4respectively.

With reference to the system block diagram FIG. 30 the four imagesensors C1 C2 C3 C4 are normal CCD image sensors except that theirsystem clock crystals have been removed and their clock input pins wiredto accept an external clock drive CLK1, CLK2, CLK3, CLK4 respectively.The composite video signals CV1, CV2, CV3, CV4 from the image sensors isfed to the Frame Line Mask FLM C24 circuitry and Combined Video CV C27circuitry.

The purpose of Frame line mask FLM C24 circuitry is to extract from eachimage sensor's composite video signal the Frame and line syncinformation L1, L2, L3, L4, waveform C34, and pass these signals to theFine Line Position FLP C26 circuitry. Further FLM C24 uses thesestripped sync signals to generate a further signal per image sensor, theFrame begin signals F1, F2, F3, and F4 waveform C33 which are passed tothe Sensor Clock Synchronisation SCS C25 circuitry.

The Sensor clock synchronisation SCS C25 circuitry controls the clockgeneration for each image sensor in such a way as to ensure thenecessary time relative frame and line sync generation of each imagesensor. In this example this is achieved by stopping individual imagesensor clocks to bring the synchronisation of all the image sensors tomeet a particular criterion. Consider that image sensor's C1synchronisation is being used as a reference and that the relativesynchronisation of image sensor C2 is required, by way of example, to be0.05 ms later while image sensor C3 in respect of image sensor C1 isrequired to be 18 ms later and image sensor C4 with respect to imagesensor C1 is required to be 18.05 ms later. If the Frame begin signal F2for image sensor C2 is delayed by 19.95 ms and that for image sensor C3F3 is delayed by 2 ms and similarly that for image sensor C4 F4 isdelayed by 1.95 ms then if the image sensors meet the requiredsynchronisation the F1, and delayed Frame begin signals F2, F3, and F4will occur simultaneously and the signal Frame LocK FLK will be set trueand remain set for a number of frame periods. Under these conditionseach image sensor will be driven continuously by a common system clock.If however, for example, image sensor C1 generates its Frame begin F1pulse before the other image sensors delayed Frame begin pulses Fin) itsclock drive is stopped, similarly the next image sensor to generate adelayed Frame begin F(n) signal has its clock stopped until all imagesensors have generated their respective delayed frame begin signals. Theinstance the last (in this case) delayed Frame begin pulse arrives allimage sensor clock drives are restarted. Synchronisation is essentiallyinstantaneous and once synchronised FLK is used to maintain a continuousclock drive to the reference image sensor however all other image sensorclock drives may be operated on when FLK is set by the circuitry of Fineline position FLP C26 circuitry.

The Fine line position FLP C26 circuitry operates on a similar principleof image sensor clock drive control using the Frame and line syncinformation L1, L2, L3, and L4 waveform C34 rather than Frame beginsignals and exercises greater sensitivity such that only single clockperiods per line scan period are used to adjust the synchronisation of aparticular image sensor in maintaining an accurate relativesynchronisation between all image sensors in the system. The feedbackloop from the Fine line position FLP C26 circuitry to the Sensor clocksynchronisation SCS C25 circuitry to achieve this functionality isthrough the Line LocK signals LLK2, LLK3, LLK4 where these signals areused in the presence of FLK to gate the clock drive CLK2, CLK3, and CLK4to image sensors C2, C3, and C4 respectively. When the Frame lock FLKsignal expires the Fine line position FLP C26 functionality is inhibitedand the image sensors relative frame synchronisation is checked within aframe period 20 ms (UK) to ensure a continuous relative synchronisationbetween the image sensors, or to restore it and in either case set FLKagain whereby fine adjustment of the image sensor synchronisation by FLPC26 can occur.

For the image sensors now synchronised it is possible to extract one ormore subsets of luminance signals from the individual image sensorscomposite video signals such that they represent the luminance signalsof one or more virtual image sensors, each having an equivalent field ofview of any component image sensor in the array. The functionalitycapable of supporting one such virtual image sensor, in this example, iscontained within the Combined video CV C27 circuitry where the one ormore subsets of luminance signal LS1, LS2, LS3, LS4 are taken from theimage sensors C1, C2, C3, and C4 forming the array and combined with anappropriate synthetically generated Frame and line sync signal ESallowing the resulting composite video signal CV5 to be displayed on theDisplay D C30. In this particular example the subsets of luminancesignal LS1, LS2, LS3, LS4 are achieved by the generation of suitablemasks to control analogue switching of the image sensor composite videosignals CV1, CV2, CV3, and CV4. The X Y outputs of a Joystick C29, orthe equivalently synthesised demands and control XYC from an externalprocessing system, control the magnitude of time constants associatedwith the generation of the vertical and horizontal luminance masksthereby effectively controlling the positioning of the virtual imagesensors field of view boresight within its field of regard. It is worthnoting that generally a rate demand would normally be more appropriateto slaving a virtual image sensor across a wide field of regard howeverin this simple example positional demands are used. The rising edge ofthe Frame begin signal F1 is used to trigger a monostable pulse Q1, withmaximum duration of one frame period and instantaneous length controlledby the Joystick C29 potentiometer Y demand, representing vertical maskinformation. The trailing edge off Q1 drives a further monostable theduration of whose output Q4 corresponds with the frame syncing period.The falling edge of stripped sync information L1 waveform C34 from thereference image sensor C1 triggers a pulse Q2 whose maximum duration isthat of a line scan period, this output, controlled by the Joystick C29X demand, represents horizontal mask information. The Q2 trailing edgetriggers a further monostable whose output Q3 and compliment Q3 are usedto generate elements of shifted sync information. Because these syncelements are generated in relation to the reference image sensor's Frameand line sync L1 signal the elevation control does not influence azimuthpositioning of the virtual image sensor's field of view boresight. Theoutput Q3 is gated with Q4 to produce shifted line sync informationwhist Q3 is gated with Q4 to give shifted frame sync information, thetwo signals being combined to form a synthetic sync signal SS which ispassed to the Video Mixer VM C28. Subsets of luminance information areextracted from one or more of the image sensors composite video signalsusing analogue gates where their switching is controlled by the verticalQ1 and horizontal Q2 mask signals according to the following: imagesensor C1 Q1&Q2, image sensor C2 Q1&Q2, image sensor C3 Q1&Q2, imagesensor C4 Q1&Q2. The four luminance subsets outputs LS1 2, 3, 4 fromimage sensors C1, C2, C3, and C4 respectively gated by the analoguegates are passed to a luminance level balancing network in the Videomixer VM C28 before being combined with the synthetic frame and linesync information SS the resulting composite video CV5 being passed tothe display D C30 or used as a front end image sensor input to a furtherprocessing system, for example an image pattern thread processor.

The essential functionality contained in the Combined video CV C27,Joystick C29 or equivalent, and Video mixer VM C28, can be replicated tosupport any number of additional autonomous virtual image sensors.

With exclusive reference to FIGS. 32-34, the operation of a three axisimage stabilization sub system is described here which utilises some ofthe generic functionality and signals introduced in earlier text. Animage sensor IS1 D1 accepts clock drive CLK from the Sensor ClockSynchronisation SCS circuitry D3 and regards a scenario where theassociated composite video CV1 is passed to two areas of functionalityFrame Line Mask FLM circuitry D4 and Analogue to Digital Converter ADCcircuitry D5.

The purpose of frame line mask FLM circuitry D4 is to strip the frameand line sync signal L from the composite video and from this strippedsync signal generate two further signals, a frame begin signal F, and animage data mask signal M, these three signals L,F,M are passed to theaddress generation ADG circuitry D6. The mask signal M is passed to theAnalogue to Digital Converter ADC circuitry D5. The frame begin signal Fis passed to the sensor clock synchronisation SCS circuitry D3.

The sensor clock synchronisation SCS circuitry D3 drives in this exampleonly one image sensor and therefore provides a continuous clock signalCLK to the image sensor IS1 1 and also feeds the signal to the ADdressGeneration ADG circuitry D6 and Analogue to Digital Converter ADCcircuitry D5.

The address generation ADG circuitry D6 accepts the clock signal CLKoperating at the maximum bandwidth frequency of the image sensor whichit gates with the mask signal M the output of which drives a series ofcascaded counters, parts of which are reset by characteristics of theframe begin F and frame and line sync L signals, and whose output is thenormal sequential address AD1 for each pixel or resolution period of theimage sensor. This address AD1 defines position in line, and line inframe of the image sensors actual scan pattern. The counters also allowframes of data to be counted, and this signal is named F0. The pixeladdress for position in line and line in frame are passed to the addresscorrection ACOR circuitry D7 and to the Read Control Logic RCL circuitryD11. F0 is passed to both the Write Control Logic WCL circuitry D8 andread control logic RCL circuitry D11 to allow the double bufferedmemories AM1 D9 and BM1 D10 to support continuous processing of imagedata.

The analogue to digital converter ADC circuitry D5 operates on thecomposite video signal CV1 gated through an analogue switch controlledby the mask signal M and converts the signal within each clock period todigital values ID1 which are passed to the write control logic WCLcircuitry D8.

With particular reference to FIG. 33 the Address CORrection ACORcircuitry D7 comprises a number of Read only Memories ROM's configuredin parallel D16 & D17, D18 & D19, D20, and D21 where further the outputsfrom specific pairs of ROM'S are cascaded through the sequence. To allowfor address component precision from the address generation ADG D6 oraxis rotation terms from the motion sensor MS D2 or to gain additionaloutput precision as here, these cascaded ROM's may be further organisedand added to stage wise in parallel. The ROM's which are continuouslyselected accept address definition AO-An and generate data DO-Dn. ROM1D16 and ROM2 D17 accept normal raster scan address definition from theADG circuitry D6 as partial address definitions, the remainder of theaddress definition comprises the roll axis correction term R.

This term may be taken from the image sensor's motion sensor MS D2,traditional accelerometer or equivalent derived velocity, or fromsoftware based pattern thread tracking algorithms in the ExternalComputer system ExCs D12 where such terms utilise advance feed-forwardtechniques to remove the latency of such calculations. The data setscontained within these ROM's, specified off line using high levelmodelling techniques, define the address transforms whose outputs DO-Dn,for specific combinations of ADG address components and axis correctionterms, represent the stable platform address components for a particularaxis rotation in this case roll. The outputs from ROM1 D16 and ROM2 D17,AR1 and AR2 respectively, form partial address definitions to ROM3 D18and ROM4 D19 respectively where the complete address definition AO-An tothese two ROM's is provided by the pitch correction term P, this term isgenerated in similar fashion by MS D2 or ExCs D12 to the roll correctionterm. Similarly the data sets contained in these ROM's are specified offline using high level modelling techniques and provide the addresstransforms, to correct for particular input combinations of the rollaxis modified ADG address term in respect of pitch. The outputs fromROM3 D18 and ROM4 D19, ARP1 and ARP2 respectively, provide the partialaddress definitions for ROM5 D20 and ROM6 D21 respectively where the yawcorrection term Y complete the address definition AO-An. The data setsfor these ROM's similarly defined by high level modelling techniquessupport the address transforms necessary for specific combinations ofaddress components to effect the yaw axis rotation of the previouslyroll and pitch corrected ADG address term. The outputs from ROM5 D20 andROM6 D21, ARPY1 and ARPY2 respectively, in this example together formthe three axis corrected address definition AD2 for the current imagesensor scan position and this complete address definition AD2 is passedto the write control logic WCL D8.

Referring again to FIG. 32 the write control logic WCL circuitry D8accepts as inputs the corrected address definition AD2 for the imagesensors current scan position, corrected for sensor motion in roll pitchand yaw, from the ACOR D7 and the digital value ID1, representing thecurrent composite video signal from the ADC D5. Further the writecontrol logic WCL D8 accepts F0 to determine which set of memories AM1D9 or BM1 D10 to use. Address and data definition to each of the doublebuffered memories is effected through two sets of three state gates oneset dedicated to the write control logic WCL D8 and one set dedicated tothe read control logic RCL D11. For the address definition AD2 from ACORD7 and data ID1 from the ADC D5 a write pulse WEA1 or WEB1 is generated,the timing of which is determined by the time necessary to achievestability of output from the ACOR D7 and ADC D5 in any image sensorresolution period.

The read control logic RCL circuitry D11 accepts as inputs F0, to allowaccess to the double buffered memories permitting reading of datawritten during the previous frame period, and the unmodified ADG D6address AD1. These inputs allow sequential reading of the memory writtenduring the previous frame. Data so read in sympathy with address AD1 maybe passed to a a further processing sub system or digital to analogueconverter DAC D13 and recombined in the video mixer VM D14 with currentstripped frame and line sync signal L to form the composite video signalCV2 which allows display of the stabilised image on display D D15. For aprocessing system with independent address generation AD3 the memoriessupport asynchronous access and processing of the stabilized data.

Referring to FIG. 34 the functionality can be seen to be similar to thatof FIG. 32 except that the image sensor's motion sensor MS D2 isreplaced by software functionality, in the external computer system ExCsD12 , which allows tracking of image pattern thread elements by accessto memories AM2 D22 and BM2 D23 providing thread data on a continuousbasis by the use of double buffering. The data for these memories isgenerated by an “image pattern thread processor” comprising a luminancedifferential processor LDF D24 or equivalent, stack pointer SP D25circuitry and the functionality of FLM D4 and ADG D6 circuitries wherein this example the luminance differential processor LDP D24 extractsimage outline and relief contour detail BV1 of the observed scenariowhich is written into the memories AM2 D22 and BM2 D23 by the writecontrol logic WCL circuitry D8. This information may be written eitheras image mapped event data using address definition AD1 and data BV1 oras in this example as vector identity lists using address definition ADGcomprising AD1 line identity component and sequential stack pointer SPcomponent P, and data specified by the AD1 address component definingposition in line VI. Asynchronous reading of the AM2 D22 and BM2 D23memories by the external computer system ExCs D12 using addressdefinition AD4 through the functionality of the read control logic RCLD11 permits specific motions of pattern thread elements to be monitoredand thereby define the roll R, pitch P, and yaw Y terms for the ACOR D7.

The functionality described allows for the three axis stabilization ofimaged data using data rate address correction applied to an imagesensors composite video signal. There is an implicit data latency of oneframe period in respect of the normal address recovery of information sobuffered.

Ideally the concept of three axis data rate address correction should beapplied inversely and upstream within the image sensor technology sothat date stored in normal address sequence as a function of acceptedoptics and CCD raster technologies may be recovered using addresscorrection on the data read out. It is recognised here that movementaway from serial to random read out of CCD information is necessary toachieve this.

In respect of virtual image sensor technologies where multiple imagesensors support a virtual imaged field of view then the integration ofstabilisation techniques into CCD technologies (as opposed to its downstream realization) requires a cooperative arrangement of addresscorrection between the associated multiple CCD elements comprising sucha virtual image sensor.

Referring exclusively to FIGS. 35-38 a more detailed description of animage pattern thread processor is now included whose genericfunctionality an d signals were largely introduced in earlier text. Withparticular reference to FIG. 35 Composite Video CV1 waveform E26 from aCCD or equivalent image sensor IS E1 observing a real world scenario E25is fed to the Frame Line Mask FLM E2 circuitry and to the LuminanceDifferential Processor LDP E3 circuitry and Display D E11.

The purpose of the Frame line mask circuitry FLM E2 is to strip from theComposite video signal CV1 E26 the Frame and line sync information L E28and pass this signal to the Video Mixer VM E4, and ADdress GenerationADG E5 circuitry. The Frame line mask FLM E2 circuitry also uses thisstripped sync information L E28 to generate two further signals a Framebegin signal F E27, and a Luminance Mask signal M E33, both of thesesignals are sent to the Address generation ADG E5 circuitry, the Masksignal M E33 is also sent to Luminance differential processor LDP E3,and Stack Pointer SP E6 circuitry.

Referring in particular to FIG. 35 the Luminance differential processorLDP E3 circuitry operates on the Composite video signal CV1 waveform E26performing spectral analysis on this signal to identify frequencyexcursions of the waveform through upper or lower settable limits. Thisis achieved simply in this example by feeding the variable gain signalfrom an emitter follower waveform E29, shown at the time slice EZEZ, totwo time differentiating CR circuits whose outputs E30 each drive a PNPtransistor E12 and E13, one E12 biased on, and the other E13 biased off.Positive going differentials (frequency excursions through an upperlimit) do not affect the transistor E12 biased on, but momentarilyswitch on the transistor E13 biased off and the output waveform E31 fromthis transistor feeds a schmitt trigger inverter E14. Similarly fornegative going differentials (frequency excursions falling below a lowerlimit) do not affect the transistor E13 biased off, but momentarilyswitches off the transistor E12 biased on the output waveform E32 feedsa double schmitt trigger inverter E15. The outputs from the invertersE14 and E15 are combined and shaped by a monostable before gating withthe Mask signal M E33 to produce the Binary Event BEC signal waveformE35 which is passed to the switch E24. Similarly the shaped componentupper and lower frequency excursion binary event signals BEU and BELwaveforms E36 and E34 respectively are passed to switch E24. Switch E24allows one of the signals BEC. BEU or BEL to be selected and passed tothe Stack pointer SP E6 circuitry, Video mixer VM E4 circuitry and WriteMemory Control WMC E7 circuitry.

The frequency characteristics of an image sensor's luminance signal areinherently polarised according to the relative alignment of the imagesensors line scan and viewed features. Mixing of the composite videosignal CV1 with a signal E21 from a clock operating at the maximumbandwidth frequency of the image sensor allows some reduction in thispolarisation for a single sensor system.

Referring again in particular to FIG. 36 the Address generation ADG E5circuitry utilises a clock operating at the maximum bandwidth frequencyof the image sensor and gated by the Mask signal M E33 the output fromwhich drives cascaded counters within each luminance period. Elements ofthe cascaded counters are reset from characteristics of the Frame beginsignal F waveform E27, and Frame and line sync signal L waveform E28,and the counters thereby generate a compound identity, equivalent toevery luminance resolution period in the image sensors raster scan, thatis for each pixel position in each line VI and every line scan within aframe SI both these signals are passed to the Write Memory Control WMCE7 circuitry. Frames of data are also counted by the Address generationADG E5 circuitry and this signal F0 is also passed to the Write memorycontrol WMC E7 circuitry, the External Computer system ExCs E8 and thedouble buffered memories MAB E9 to allow synchronised use of thesememories by WMC E7 and ExCs E8. The Address generation ADG E5 circuitryalso generates an End Marker signal EM E37 which indicates the end ofany particular luminance period and this signal is passed to the Stackpointer SP E6 circuitry and the Write memory control WMC E7 circuitry.

Referring in particular to to FIG. 37 which shows the organisation ofthe MAB E9 memories comprising the pair of physically and logicallypartitioned memories M1A E50 and M1B E51. The memories M1A E50 and M1BE51 are sized according to the number of line scans in a frame, andresolution periods in a line scan. The memories are capable ofregistering data written within the resolution period of the imagesensor. The memory address and data lines are accessed via sets of threestate gates one set E52, E53, E54, E55 dedicated to the Write memorycontrol WMC E7 and the other set E56, E57, E58, E59 to the read memorycontrol of the External computer system ExCs E8. These memories arecontinuously selected and during each frame data is written by WMC E7and read by the External comouter system ExCs E8 these processes beingperformed one frame out of phase for each of the memories M1A E50 andM1B E51. The Write memory control WMC E7 generates a write address WA1and data WD1 which are presented to the sets of three state gates E52,E54, and E53, E55 respectively. During frames when OF=1 the outputs fromthe gates E54 and E55 are enabled and drive memory M1B E51, write enablepulses WEIB are generated by WMC E7. For the read addresses RA1presented by the read memory control of the External computer systemExCs E8 to the sets of three state gates E56 and E58 only the gates E56are enabled for frames when F0=1 and during these frame periods data RD1in response to the OEIA signals generated by the External computersystem ExCs E8 is returned through the enabled three state gates E57. Onthe subsequent cycle write addresses WA1 and data WD1 are presented toM1A via the sets of enabled three state gates E52 and E53 respectivelywhile data RD1 is read from M1B using the enabled three state gates E59corresponding to the read addresses RA1 presented via the enabled threestate gates E58.

Referring again in particular to FIG. 35 the Stack pointer SP E6circuitry accepts the End marker signal EM waveform E37 and the switchE24 selected binary event signal BEC waveform E35, or BEU waveform E36,or BEL waveform E34 and counts events within a line scan to allowsequential address generation P within logically partitioned stacksdefined within the memories M1A E50 and M1B E51, where each new addressis the address into which the next entry in any particular stack will bemade. The stack pointer P is reset at the beginning of each new linescan by the action of the Mask signal M E33 going low.

For data compression of image pattern thread data the Write memorycontrol WMC E7 has at any particular time the identity of the availablememory M1A E50 or M1B E51 decided in this case by the Frame OF signal,the current logically partitioned stack base within memory SI (linenumber within the frame), it also has the vector identity VI(representing the position in a line scan of any possible luminancefrequency excursion binary event). Further during any particular linescan the address P from the Stack pointer SP E6 circuitry indicates thesequential word in a particular stack to be written. When a frequencyexcursion event occurs the associated binary event in the signal BECwaveform E35 or BEL waveform E34, or BEU waveform E36 depending onswitch E24 position is used by the Write memory control WMC E7 circuitryto generate the WE for the memory in question for this particular frameperiod, when the data WD1 written into the word, indicated by theaddress combination WA1, formed from the stack identity SI and the stackaddress P, will be the vector identity VI (WA1=SI+P WD1=VI). At the endof each line's luminance period the end marker signal EM E37 will beinterpreted as though it were a binary event and will cause a word WD1to be written to the current stack whose identity is unique andindicative on reading that the logical end of a particular stack of datahas been reached. During alternate frames stacks written by the Writememory control WMC E7 circuitry in the previous frame may be readasynchronously by the External computer system ExCs E8.

For the writing of image mapped binary event data, the Write memorycontrol WMC E7 circuitry again identifies the appropriate memory towrite to, for the current frame, on the basis of F0 but now generates anappropriate write enable command WE1A or WE1B for every addressWA1=SI+VI with data WD1=(BEC or BEU or BEL) as a function of the switchE24 setting. Continuous asynchronous access to data written during theprevious frame can be made by the External computer system ExCs E8.

The Video Mixer VM E4 combines Binary event data BEC, BEU, or BEL(function of switch E24) with Frame and line sync signal L E28 to form acomposite video signal CV2 which is passed to the Display D E11 allowingoperator observation of the image pattern thread processor output E38for lower frequency excursion events, E39 for combined upper and lowerfrequency excursion events, and E40 for upper frequency excursionevents. The Display D E11 also allows viewing of composite video CV1representing the normal video image E25 waveform E26.

What is claimed is:
 1. A topography processor system arranged to execute internal transfer functions, the system comprising, an array of at least two image systems, and at least one processor, wherein the processor includes processing system diagnostic means for stimulating the system with predefined data, wherein the processor is arranged to process imaged detail on said predefined data and generate for output at least one of a graphic visual, augmented visual, and automatic visual display, and wherein the processing system diagnostic means is arranged to diagnose system faults and internal transfer function faults by employing at least one of the graphic visual, augmented visual, and automatic visual displays.
 2. A topography processor system as claimed in claim 1 wherein a frame rate topography processor is provided which allows stabilised frame rate macroscopic system performance.
 3. A topography processor system as claimed in claim 1 wherein said diagnosis is performed non interruptively to allow an operationally configured system and its performance envelope to be preserved.
 4. A topography processor system as claimed in claim 3 wherein the non interruptive aspect of the diagnosis does not require extensive second layer BITE.
 5. A topography processor system as claimed in claim 3 wherein the diagnostic monitoring is possible using system sufficient data, control and address highways.
 6. A topography processor system as claimed in claim 3 wherein the diagnostic monitoring does not require additional system external equipments.
 7. A topography processor system as claimed in claim 1 wherein the diagnosis provided by said diagnostic means encompasses all the system processes executing to completion.
 8. A topography processor system as claimed in claim 1 wherein the diagnosis allows first line system GO/NOGO decisions to be reached, based on the performance of system transfer functions.
 9. A topography processor system as claimed in claim 1 further including augmented visual diagnostic means to allow first line monitoring of system end to end performance against apriori reference data.
 10. A topography processor system as claimed in claim 1 further including automatic visual means to allow first line monitoring of system end to end performance against apriori reference data.
 11. A topography processor system as claimed in claim 1 further including non interruptive system stimulation and monitoring means for system performance analysis.
 12. A topography processor system as claimed in claim 1 further including correlatable differential image sensor input means to allow stimulation of the topography processor system.
 13. A topography processor system as claimed in claim 1 further including closed loop stimulation and monitoring means to allow system performance assessment.
 14. A topography processor system as claimed in claim 1 further comprising resolution means for employing an association of generic patterns between a plurality of image sensors to resolve the system relative spacial position of topographical detail in the observed scenario.
 15. A topography processor system as claimed in claim 1 further comprising identification and partitioned storage means which allow binary event signals comprising generic image pattern information, output from an image pattern thread processor or equivalent and representative of vectors of particular and different attributes, to be generated from each of the image sensors composite video signals and for each binary event comprising such signals to assign an identity and pass such information as partitioned sets of data to other processes.
 16. A topography processor system as claimed in claim 1 further comprising combination synthesis means which allows combinations of vector identities from sets of sets of such identities to be synthesised and passed to other processes.
 17. A topography processor system as claimed in claim 1 further comprising identity transform means which support parallel and cascaded operations on sets of synthesised identity combinations where apriori transform knowledge is employed to solve vector equations or other processes to be computed within an effective multiple address time.
 18. A topography processor system as claimed in claim 1 further comprising dedicated ordered partitioned buffer means to register results from parallel identity transform processes and allows, through the ordered reading of sets of such buffers, the output of a three dimensional scenario data base, comprising unique multiple vector intercepts.
 19. A topography processor system as claimed in claim 1 further comprising processor architecture means to allow dynamic reconfiguration of processor resources necessary to support frame rate operation.
 20. A topography processor system as claimed in claim 1 further comprising means for co-ordinated scanning of a common scenario by the image sensors whilst maintaining their different perspectives to allow improved range discrimination.
 21. A topography processor system as claimed in claim 1 further comprising means for iterative synthesis of combinations of, and comparison of, subset patterns between sets of such subset patterns derived from a plurality of image sensors composite video signals, where correlation of such subset patterns members between image sensors is detected, thereby to allow the system relative spacial position of their associated elements of topographical detail to be fixed from apriori knowledge.
 22. A topography processor system as claimed in claim 1 further comprising real time identification and partitioning means to extract subset patterns from a plurality of composite video signals.
 23. A topography processor system as claimed in claim 22 wherein information of identified and partitioned subset patterns may be passed to a further process.
 24. A topography processor system as claimed in claim 1 further comprising electro-mechanical means to allow iterative synthesis of subset pattern combinations having time coincidence between a plurality of image sensors.
 25. A topography processor system as claimed in claim 1 further comprising relative image sensor frame and line synchronisation means to allow iterative synthesis of subset pattern combinations having time coincidence between a plurality of image sensors.
 26. A topography processor system as claimed in claim 1 further comprising control loop means to refine iterative synthesis of combinations of subset patterns having time coincidence between a plurality of image sensors.
 27. A topography processor system as claimed in claim 1 further comprising real time multiple comparison means to identify unique multiple matches between sets of sets of subset patterns.
 28. A topography processor system as claimed in claim 1 further comprising real time computation means to determine the system relative spacial position of elements of topographical detail defined by a multiple match of subset pattern members between image sensors.
 29. A topography processor system as claimed in claim 1 further comprising means for passing three dimensional scenario data base to other processes.
 30. A topography processor system as claimed in claim 1 wherein co-operatively slaveable phased virtual image sensor array means are provided to allow a plurality of equivalent virtual image sensors, to be logically and physically positioned and orientated, such that the boresights of their respective fields of view are parallel and such that they generate images of a common scenario from their different perspectives, and that common image detail is registered in corresponding line scans for each such virtual image sensor, and that the frame and line sync generation for each such sensor is controlled such that a time coincidence exists between the characteristics of theses signals, and each such virtual image sensor's field of view may be slaved or scan in a controlled fashion so as to preserve their differing perspective views of a common scenario within their shared field of regard, and such that imaged information may be passed to a further process.
 31. A topography processor system as claimed in claim 1 further comprising control means for autonomous automatic or manual definition of a virtual image sensor's field of view boresight slave position within the field of regard of a phased virtual image sensor array.
 32. A topography processor system as claimed in claim 1 further comprising control means for an external processor defined slave position of a virtual image sensor's field of view boresight within the field of regard of a phased virtual image sensor array.
 33. A topography processor system as claimed in claim 1 further comprising control means for an autonomous iterative inter frame time relative shift of a frame and line sync between a plurality of image sensors in a phased image sensor array.
 34. A topography processor system as claimed in claim 1 further comprising control means to allow for an external processor defined time relative shift of a frame and line sync separation between a plurality of image sensors in a phased image sensor array.
 35. A topography processor system as claimed in claim 1 further comprising communication means for allowing imaged information derived from imagers comprising a phased image sensor array, to be passed to an external processor system.
 36. A topography processor system as claimed in claim 1 further comprising virtual image sensor means to allow a plurality of similar image sensors to be organised in an array, where the logical and physical position and orientation of each such image sensor in the array is such that their individual fields of view collectively cover a continuous scenario comprising the individual images from each image sensor, and where it is possible to generate in real time a virtual image sensor image from components of images from one or more adjoining image sensors in the array such that the field of view of the virtual image sensor is equivalent to the field of view of any image sensor in the array and where the field of regard of the virtual image sensor comprises the individual fields of view of the image sensors in the array.
 37. A topography processor system as claimed in claim 1 further comprising relative synchronisation means for establishing and maintaining a relative frame and line sync separation between image sensors comprising a virtual image sensor necessary to allow real time extraction of component image sensor luminance subsets in support of a virtual image.
 38. A topography processor system as claimed in claim 37 further comprising control means to define the virtual image sensor's field of regard relative virtual image sensor's boresight position.
 39. A topography processor system as claimed in claim 37 further comprising logical means to identify luminance signal subsets and frame and line sync characteristics for each virtual image sensor's field of view included in the system.
 40. A topography processor system as claimed in claim 1 further comprising composite video synthesis means for combining subsets of luminance signals with an appropriately generated synthetic frame and line sync signal to generate a composite video signal of each virtual image sensor included in the system.
 41. A topography processor system as claimed in claim 1 further comprising functional replication means for a given array of image sensors comprising a virtual image sensor included in the system to support the generation of multiple, simultaneous and different virtual image sensor composite video signals.
 42. A topography processor system as claimed in claim 1 further comprising manual, automatic or external processor definition means to electronically slave one or more virtual image sensors field or fields of view independently and simultaneously across a field of regard comprising the fields of view of the image sensors comprising the array.
 43. A topography processor system as claimed in claim 1 further comprising electronic means for the rapid and accurate positioning of a field of view boresight of a virtual image sensor within its field of regard.
 44. A topography processor system as claimed in claim 1 further comprising communication means to allow information in a virtual image sensor's field of view to be communicated to another processing system.
 45. A topography processor system as claimed in claim 1 further comprising three axis image stabilization means employing address correction applied to the buffering of information derived from an image sensor composite video signal, to allow stabilized data to be recovered using normal addressing.
 46. A topography processor system as claimed in claim 1 further comprising three axis image stabilization means employing address correction applied to allow the recovery of stabilised image sensor luminance data from buffered information derived from an image sensors composite video signal, stored using normal addressing.
 47. A topography processor system as claimed in claim 1 further comprising three axis image stabilization means employing address correction terms based on information from traditional rate sensors.
 48. A topography processor system as claimed in claim 1 further comprising three axis image stabilization means employing address correction terms based on information from software pattern tracking algorithms.
 49. A topography processor system as claimed in claim 1 further comprising three axis image stabilization means employing address correction applied to allow the recovery of stabilised image sensor luminance data from buffered information derived from an image sensors sensed luminance, stored using CCD or equivalent technologies.
 50. A topography processor system as claimed in claim 1 further comprising three axis image stabilization for a plurality of associated (in the context of virtual image sensor or phased image sensor arrays) CCD sensing means employing address correction applied to allow the recovery of stabilised image sensor luminance data from buffered information derived from an image sensors sensed luminance, stored using CCD or equivalent technologies.
 51. A topography processor system as claimed in claim 1 further comprising image pattern thread processing means which allow for the real time extraction of important image pattern thread information from the composite video signal of an image sensor, CCD or equivalent, comprising a luminance signal spectral analyser or equivalent capable of identifying image pattern outline and relief contour detail derived from luminance signal frequency excursions transiting through preset upper or lower frequency limits where such events generate in real time a binary event signal and an associated identity for each such event and where such information may be passed to a further processing or display system.
 52. A topography processor system as claimed in claim 1 further comprising real time identification means to log the position within the image sensor's raster scan, that is within the field of view of the image sensor, of each binary event representing elements of image pattern threads.
 53. A topography processor system as claimed in claim 1 further comprising real time identification means to count the sequential binary events of particular attributes within each line scan of the image sensor.
 54. A topography processor system as claimed in claim 1 further comprising communication means to pass to further processing systems partitioned sequential terminated lists of binary event identities representative of imaged pattern thread elements.
 55. A topography processor system as claimed in claim 1 further comprising communication means to pass to further processing systems binary event data as an image mapped array.
 56. A topography processor system as claimed in claim 1 further comprising output double buffering of image pattern thread data means to pass such data to a further processing system allowing continuous data output at an image sensor frame rate with maximum data latencies of 20 ms (UK).
 57. A topography processor system as claimed in claim 1 further comprising data reduction means to expedite the further analysis of image pattern thread data.
 58. A topography processor system as claimed in claim 1 further comprising interference means which through mixing of a clock signal operating at the maximum bandwidth frequency of the image sensor with luminance information allows a reduction in polarisation effects of relative imaged feature and image sensor line scan orientation.
 59. A topography processor system as claimed in claim 1 further comprising means for the display of composite video synthesised by the combination of the binary event signal with current stripped frame and line sync information from a composite video signal of the image sensors.
 60. A topography processor system as claimed in claim 1 further comprising a variety of processors to simultaneously identify image pattern thread elements with differing attributes for simultaneous output to a further processing or display system. 